The Living Thing / Notebooks :

Density estimation as count regression

We can formulate density estimation as a count regression; For “nice” distributions this will be the same as where the target is estimating the correct Poisson intensity for any given small region of the state space. (e.g. Gu93 or EiMa96)

In the nonparametric case this can be your basic functional data problem, although I’m not quite sure how the integrates-to-unity thing is handled. If it’s not, probably it’s linear point process regression/ Possibly we ignore the requirement to integrate to unity unless we need it (as in mixture models)

TBD: which precisely are the nice distributions? I suspect it will be necessary that the samples viewed as a spatial point process are simple which means, e.g. no atoms. Other criteria? See Scho05 for this niceness question in a regression setting.

Questions

  1. What does general functional regression get me here? (see Ch21 of Ramsay and Silverman)
  2. Can I use point process estimation theory to improve density estimation? After all, normal point-process estimation claims to be an un-normalised vesion of density estimation. Lies11 draws some parallels there, esp. with mixture models.

Refs

BeDi89
Berman, M., & Diggle, P. (1989) Estimating Weighted Integrals of the Second-Order Intensity of a Spatial Point Process. Journal of the Royal Statistical Society. Series B (Methodological), 51(1), 81–92.
BrCZ10
Brown, L. D., Cai, T. T., & Zhou, H. H.(2010) Nonparametric regression in exponential families. The Annals of Statistics, 38(4), 2005–2046. DOI.
Cox65
Cox, D. R.(1965) On the Estimation of the Intensity Function of a Stationary Point Process. Journal of the Royal Statistical Society. Series B (Methodological), 27(2), 332–337.
CuSS08
Cunningham, J. P., Shenoy, K. V., & Sahani, M. (2008) Fast Gaussian process methods for point process intensity estimation. (pp. 192–199). ACM Press DOI.
Efro96
Efromovich, S. (1996) On nonparametric regression for IID observations in a general setting. The Annals of Statistics, 24(3), 1126–1144. DOI.
Efro07
Efromovich, S. (2007) Conditional density estimation in a regression setting. The Annals of Statistics, 35(6), 2504–2535. DOI.
EiMa96
Eilers, P. H. C., & Marx, B. D.(1996) Flexible smoothing with B-splines and penalties. Statistical Science, 11(2), 89–121. DOI.
Elli91
Ellis, S. P.(1991) Density estimation for point processes. Stochastic Processes and Their Applications, 39(2), 345–358. DOI.
GiKM08
Giesecke, K., Kakavand, H., & Mousavi, M. (2008) Simulating point processes by intensity projection. In Simulation Conference, 2008. WSC 2008. Winter (pp. 560–568). DOI.
Gu93
Gu, C. (1993) Smoothing Spline Density Estimation: A Dimensionless Automatic Algorithm. Journal of the American Statistical Association, 88(422), 495–504. DOI.
Nore10
Norets, A. (2010) Approximation of conditional densities by smooth mixtures of regressions. The Annals of Statistics, 38(3), 1733–1766. DOI.
PaZe16
Panaretos, V. M., & Zemel, Y. (2016) Separation of Amplitude and Phase Variation in Point Processes. The Annals of Statistics, 44(2), 771–812. DOI.
Papa74
Papangelou, F. (1974) The conditional intensity of general point processes and an application to line processes. Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete, 28(3), 207–226. DOI.
Reyn03
Reynaud-Bouret, P. (2003) Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities. Probability Theory and Related Fields, 126(1). DOI.
Scho05
Schoenberg, F. P.(2005) Consistent parametric estimation of the intensity of a spatial–temporal point process. Journal of Statistical Planning and Inference, 128(1), 79–93. DOI.
STSK10
Sugiyama, M., Takeuchi, I., Suzuki, T., Kanamori, T., Hachiya, H., & Okanohara, D. (2010) Conditional density estimation via least-squares density ratio estimation. In International Conference on Artificial Intelligence and Statistics (pp. 781–788).
Lies11
van Lieshout, M.-C. N. M.(2011) On Estimation of the Intensity Function of a Point Process. Methodology and Computing in Applied Probability, 14(3), 567–578. DOI.
WiNo07
Willett, R. M., & Nowak, R. D.(2007) Multiscale Poisson Intensity and Density Estimation. IEEE Transactions on Information Theory, 53(9), 3171–3187. DOI.