# Multivariate convex regression: global risk bounds and adaptation

@article{Han2016MultivariateCR, title={Multivariate convex regression: global risk bounds and adaptation}, author={Qiyang Han and Jon A. Wellner}, journal={arXiv: Statistics Theory}, year={2016} }

We study the problem of estimating a multivariate convex function defined on a convex body in a regression setting with random design. We are interested in optimal rates of convergence under a squared global continuous $l_2$ loss in the multivariate setting $(d\geq 2)$. One crucial fact is that the minimax risks depend heavily on the shape of the support of the regression function. It is shown that the global minimax risk is on the order of $n^{-2/(d+1)}$ when the support is sufficiently smooth… Expand

#### Figures from this paper

#### 38 Citations

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

- Mathematics
- 2020

The least squares estimator (LSE) is shown to be suboptimal in squared error loss in the usual nonparametric regression model with Gaussian errors for $d \geq 5$ for each of the following families of… Expand

Optimality of Maximum Likelihood for Log-Concave Density Estimation and Bounded Convex Regression

- Mathematics, Computer Science
- 2019

It is proved that estimating a log-concave density - even a uniform distribution on a convex set - up to a fixed accuracy requires the number of samples at least exponential in the dimension. Expand

Multivariate Convex Regression at Scale

- Mathematics, Computer Science
- 2020

This framework can solve instances of the convex regression problem with $n=10^5$ and $d=10$---a QP with 10 billion variables---within minutes; and offers significant computational gains compared to current algorithms. Expand

Adaptation in log-concave density estimation

- Mathematics
- 2016

The log-concave maximum likelihood estimator of a density on the real line based on a sample of size $n$ is known to attain the minimax optimal rate of convergence of $O(n^{-4/5})$ with respect to,… Expand

Max-affine regression with universal parameter estimation for small-ball designs

- Mathematics, Computer Science
- 2020 IEEE International Symposium on Information Theory (ISIT)
- 2020

This paper shows that AM is significantly more robust than the setting of [1]: It converges locally under small-ball design assumptions, and even when the underlying parameters are chosen with knowledge of the realized covariates. Expand

On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression

- Mathematics
- Journal of the American Statistical Association
- 2019

Abstract–In this article, we consider the nonparametric regression problem with multivariate predictors. We provide a characterization of the degrees of freedom and divergence for estimators of the… Expand

Multivariate Distributionally Robust Convex Regression under Absolute Error Loss

- Computer Science, Mathematics
- NeurIPS
- 2019

A novel non-parametric multidimensional convex regression estimator designed to be robust to adversarial perturbations in the empirical measure is proposed, matching the bounds of alternative estimators based on square-loss minimization. Expand

Adaptation in multivariate log-concave density estimation

- Mathematics
- 2020

We study the adaptation properties of the multivariate log-concave maximum likelihood estimator over two subclasses of log-concave densities. The first consists of densities with polyhedral support… Expand

Convex Regression: Theory, Practice, and Applications

- Computer Science
- 2016

The new excess risk upper bound is developed for the general empirical risk minimization setting without any shape constraints, and provides a probabilistic guarantee for cases with unbounded hypothesis classes, targets, and noise models. Expand

Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation

- Computer Science, Mathematics
- ArXiv
- 2019

This work analyzes a natural alternating minimization (AM) algorithm for the non-convex least squares objective and shows that the AM algorithm, when initialized suitably, converges with high probability and at a geometric rate to a small ball around the optimal coefficients. Expand

#### References

SHOWING 1-10 OF 62 REFERENCES

Global risk bounds and adaptation in univariate convex regression

- Mathematics
- 2013

We consider the problem of nonparametric estimation of a convex regression function $$\phi _0$$ϕ0. We study the risk of the least squares estimator (LSE) under the natural squared error loss. We show… Expand

Global rates of convergence in log-concave density estimation

- Mathematics
- 2014

The estimation of a log-concave density on $\mathbb{R}^d$ represents a central problem in the area of nonparametric inference under shape constraints. In this paper, we study the performance of… Expand

An Improved Global Risk Bound in Concave Regression

- Mathematics
- 2015

A new risk bound is presented for the problem of convex/concave function estimation, using the least squares estimator. The best known risk bound, as had appeared in \citet{GSvex}, scaled like… Expand

Sharp oracle inequalities for Least Squares estimators in shape restricted regression

- Mathematics
- 2015

The performance of Least Squares (LS) estimators is studied in isotonic, unimodal and convex regression. Our results have the form of sharp oracle inequalities that account for the model… Expand

On risk bounds in isotonic and other shape restricted regression problems

- Mathematics
- 2015

We consider the problem of estimating an unknown $\theta\in {\mathbb{R}}^n$ from noisy observations under the constraint that $\theta$ belongs to certain convex polyhedral cones in ${\mathbb{R}}^n$.… Expand

Model selection for regression on a random design

- Mathematics
- 2002

We consider the problem of estimating an unknown regression function when the design is random with values in . Our estimation procedure is based on model selection and does not rely on any prior… Expand

A new perspective on least squares under convex constraint

- Mathematics
- 2014

Consider the problem of estimating the mean of a Gaussian random vector when the mean vector is assumed to be in a given convex set. The most natural solution is to take the Euclidean projection of… Expand

Entropy of Convex Functions on ℝ d.

- Mathematics, Medicine
- Constructive approximation
- 2017

The results imply that the universal lower bound ε-d/2 is also an upper bound for all d-polytopes, and the universal upper bound of [Formula: see text] is attained by the closed unit ball. Expand

Consistency of Multidimensional Convex Regression

- Mathematics, Computer Science
- Oper. Res.
- 2012

This paper studies a least-squares estimator that is computable as the solution of a quadratic program and establishes that it converges almost surely to the “true” function as n → ∞ under modest technical assumptions. Expand

Risk bounds for model selection via penalization

- Mathematics
- 1999

Abstract Performance bounds for criteria for model selection are developed using recent theory for sieves. The model selection criteria are based on an empirical loss or contrast function with an… Expand