The Living Thing / Notebooks :

Density estimation as count regression

We can formulate density estimation as a count regression; For “nice” distributions this will be the same as where the target is estimating the correct Poisson intensity for any given small region of the state space. (e.g. Gu93 or EiMa96)

In the nonparametric case this can be your basic functional data problem, although I’m not quite sure how the integrates-to-unity thing is handled. If it’s not, probably it’s linear point process regression/ Possibly we ignore the requirement to integrate to unity unless we need it (as in mixture models)

TBD: which precisely are the nice distributions? I suspect it will be necessary that the samples viewed as a spatial point process are simple which means, e.g. no atoms. Other criteria? See Scho05 for this niceness question in a regression setting.


  1. What does general functional regression get me here? (see Ch21 of Ramsay and Silverman)
  2. Can I use point process estimation theory to improve density estimation? After all, normal point-process estimation claims to be an un-normalised vesion of density estimation. Lies11 draws some parallels there, esp. with mixture models.


Berman, M., & Diggle, P. (1989) Estimating Weighted Integrals of the Second-Order Intensity of a Spatial Point Process. Journal of the Royal Statistical Society. Series B (Methodological), 51(1), 81–92.
Brown, L. D., Cai, T. T., & Zhou, H. H.(2010) Nonparametric regression in exponential families. The Annals of Statistics, 38(4), 2005–2046. DOI.
Cox, D. R.(1965) On the Estimation of the Intensity Function of a Stationary Point Process. Journal of the Royal Statistical Society. Series B (Methodological), 27(2), 332–337.
Cunningham, J. P., Shenoy, K. V., & Sahani, M. (2008) Fast Gaussian process methods for point process intensity estimation. (pp. 192–199). ACM Press DOI.
Efromovich, S. (1996) On nonparametric regression for IID observations in a general setting. The Annals of Statistics, 24(3), 1126–1144. DOI.
Efromovich, S. (2007) Conditional density estimation in a regression setting. The Annals of Statistics, 35(6), 2504–2535. DOI.
Eilers, P. H. C., & Marx, B. D.(1996) Flexible smoothing with B-splines and penalties. Statistical Science, 11(2), 89–121. DOI.
Ellis, S. P.(1991) Density estimation for point processes. Stochastic Processes and Their Applications, 39(2), 345–358. DOI.
Giesecke, K., Kakavand, H., & Mousavi, M. (2008) Simulating point processes by intensity projection. In Simulation Conference, 2008. WSC 2008. Winter (pp. 560–568). DOI.
Gu, C. (1993) Smoothing Spline Density Estimation: A Dimensionless Automatic Algorithm. Journal of the American Statistical Association, 88(422), 495–504. DOI.
Norets, A. (2010) Approximation of conditional densities by smooth mixtures of regressions. The Annals of Statistics, 38(3), 1733–1766. DOI.
Panaretos, V. M., & Zemel, Y. (2016) Separation of Amplitude and Phase Variation in Point Processes. The Annals of Statistics, 44(2), 771–812. DOI.
Papangelou, F. (1974) The conditional intensity of general point processes and an application to line processes. Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete, 28(3), 207–226. DOI.
Reynaud-Bouret, P. (2003) Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities. Probability Theory and Related Fields, 126(1). DOI.
Schoenberg, F. P.(2005) Consistent parametric estimation of the intensity of a spatial–temporal point process. Journal of Statistical Planning and Inference, 128(1), 79–93. DOI.
Sugiyama, M., Takeuchi, I., Suzuki, T., Kanamori, T., Hachiya, H., & Okanohara, D. (2010) Conditional density estimation via least-squares density ratio estimation. In International Conference on Artificial Intelligence and Statistics (pp. 781–788).
van Lieshout, M.-C. N. M.(2011) On Estimation of the Intensity Function of a Point Process. Methodology and Computing in Applied Probability, 14(3), 567–578. DOI.
Willett, R. M., & Nowak, R. D.(2007) Multiscale Poisson Intensity and Density Estimation. IEEE Transactions on Information Theory, 53(9), 3171–3187. DOI.