In this work, we are trying to improve upon the prior works Wong et al.(Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope) and Zhang et al.(Efficient Neural Network Robustness Certification with General Activation Functions) of convex linear approximation of ReLU networks to obtain certified adversarial perturbations. Existing work tries to get a certified lower bound on the minimal adversarial bound using convex relaxations for ReLU networks. For a fixed normed $\epsilon$-ball for an input instance $x$ i.e. $\mathcal{B}(x,\epsilon)$, they strive to find linear approximations for ReLU activation functions. This convex relaxation is used to upper and lower bound the components of the logit(final) layer. Although the linear relaxation thus used, helps in finding certified lower bound on the minimal adversarial bounds efficiently, it is at the cost of large error in approximating the ReLU function. Our work addresses this issue and builds on the work based on convex relaxations. To minimize the error in approximating the ReLU activations, we use regulated softplus function which we call ReST. Our approximation is based on adaptive selection of ReST, Linear and Quadratic bounds for ReLU activations in such a manner that we obtain concave upper bound and convex lower bound for the component functions of the logit layer. Thus, for a targeted attack, we obtain a concave objective to be maximized till a threshold.