Resources on Manifold Learning (1)

Manifold Learning

Copied from lawhiu@cse.msu.edu., http://archive.itee.uq.edu.au/~huang/research_manifold.htm

Papers

ISOMAP and related

• Seeing through water. Alexei Efros, Volkan Isler, Jianbo Shi, Mirko Visontai. NIPS 2004
• O. Jenkins and M. Mataric. A Spatio-temporal Extension to Isomap Nonlinear Dimension Reduction. Proceedings of the Twenty-First International Conference on Machine Learning (ICML-2004), July 4-8, 2004, Banff, Alberta, Canada. [Abstract]
• J. Costa and A. O. Hero, Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Transactions on Signal Processing, vol. 25, no. 8, pp. 2210-2221, August 2004.
• M. H. Law, N. Zhang, A. K. Jain. Nonlinear Manifold Learning for Data Stream. Proceedings of SIAM Data Mining, pp. 33-44, Orlando, Florida, 2004.
• H. Zha and Z. Zhang. Isometric Embedding and Continuum ISOMAP. Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), Washington, DC on August 21-24, 2003.
• V. de Silva and J. B. Tenenbaum. Global versus local methods in nonlinear dimensionality reduction . Neural Information Processing Systems 15 (NIPS’2002), pp. 705-712, 2003.
• F. M´emoli and G. Sapiro. Distance Functions and Geodesics on Points Clouds . Institute for Mathematics and its Applications, Dec 2002.
• N.A. Laskaris, A.A. Ioannides. Semantic geodesic maps: a unifying geometrical approach for studying the structure and dynamics of single trial evoked responses. Clinical Neurophysiology, 113 (2002) 1209¨C1226
• M. Balasubramanian, E. L. Schwartz, J. B. Tenenbaum, V. de Silva and J. C. Langford. The Isomap Algorithm and Topological Stability . Science, vol. 295(5552), 7a, 2002.
• V. de Silva, J.B. Tenenbaum. Unsupervised learning of curved manifolds. Nonlinear Estimation and Classification, 2002, Springer-Verlag, New York.
• D. L. Donoho and C. Grimes. When Does ISOMAP Recover Natural Parameterization of Families of Articulated Images? Technical Report 2002-27, Department of Statistics, Stanford University, Aug 2002. (abstract)
• J. A. Lee, A. Lendasse and M. Verleysen. Curvilinear Distance Analysis versus Isomap ESANN 2002 proceedings – European Symposium on Artifical Neural Networks , Bruges (Belgium), 24-26 April 2002, pp.185–192.
• J.B. Tenenbaum, V. de Silva and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, vol. 290, pp. 2319–2323, 2000.
• M. Bernstein, V. de Silva, J. Langford, and J. Tenenbaum. Graph Approximations to Geodesics on Embedded Manifolds. Technical Report, Department of Psychology, Stanford University, 2000.

LLE and related

• H. Chang, D.Y. Yeung, Y. Xiong. Super-resolution through neighbor embedding. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol.1, pp.275-282, Washington, DC, USA, 27 June – 2 July 2004. (PDF)
• D. de Ridder and M. Loog and M.J.T. Reinders. Local Fisher embedding. Proc. 17th International Conference on Pattern Recognition (ICPR2004), 2004.
• L. K. Saul and S. T. Roweis. Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds.
  Journal of Machine Learning Research, v4, pp. 119-155, 2003.
• Zhenyue Zhang and Hongyuan Zha. Local Linear Smoothing for Nonlinear Manifold Learning . CSE-03-003, Technical Report, CSE, Penn State Univ., 2003.
• de Ritter D, Kouropteva O, Okun O, Pietikäinen M & Duin RPW. Supervised locally linear embedding. Artificial Neural Networks and Neural Information Processing, ICANN/ICONIP 2003 Proceedings, Lecture Notes in Computer Science 2714, Springer, 333-341.
• Kouropteva O, Okun O & Pietikäinen M (2003) Classification of handwritten digits using supervised locally linear embedding algorithm and support vector machine. Proc. of the 11^th European Symposium on Artificial Neural Networks (ESANN’2003), April 23-25, Bruges, Belgium, 229-234. Full paper
• Hadid A & Pietikäinen M (2003). Efficient locally linear embeddings of imperfect manifolds. Proc. Machine Learning and Data Mining in Pattern Recognition. Lecture Notes in Computer Science 2734, Springer, 188-201
• Kouropteva O, Okun O, Hadid A, Soriano M, Marcos S & Pietikäinen M (2002) Beyond Locally Linear Embedding Algorithm. Technical Report MVG-01-2002, University of Oulu, Machine Vision Group, Information Processing Laboratory, 49 p. Full paper
• D. De Ridder and Duin, R.P.W. Locally linear embedding for classification, Technical report PH-2002-01, Pattern Recognition Group, Dept. of Imaging Science & Technology, Delft University of Technology, pp. 1-15, 2002.
• Kouropteva O, Okun O & Pietikäinen M (2002) Selection of the optimal parameter value for the locally linear embedding algorithm. Proc. of the 1 ^st International Conference on Fuzzy Systems and Knowledge Discovery (FSKD’02), November 18-22, Singapore, 359-363. Full paper
• P. Perona and M. Polito. Grouping and dimensionality reduction by locally linear embedding. Neural Information Processing Systems 14 (NIPS’2001).
• D. DeCoste. Visualizing Mercer Kernel Feature Spaces Via Kernelized Locally-Linear Embeddings. The 8th International Conference on Neural Information Processing (ICONIP2001), November 2001.
• S. T. Roweis and L. K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding . Science, vol. 290, pp. 2323–2326, 2000.

Laplacian Eigenmap and other related

See also graph spectral methods.

• Xiaofei He; Shuicheng Yan; Yuxiao Hu; Niyogi, P.; Hong-Jiang Zhang. Face Recognition Using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence Volume: 27, number 3, March 2005, pp 328- 340 [IEEExplore]
• M.Belkin, P. Niyogi, Semi-supervised Learning on Manifolds , Machine Learning Journal, Special Issue on Clustering, to appear.
• M. Belkin, P. Niyogi, Laplacian Eigenmaps for Dimensionality Reduction and Data Representation, Neural Computation, June 2003; 15 (6):1373-1396.
• X. He, P. Niyogi. Locality Preserving Projections. Neural Information Processing Systems 16 (NIPS’2003).
• M. Brand. Nonlinear dimensionality reduction by kernel eigenmaps. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 547-552, Acapulco, Mexico, 9-15 August 2003.
• M. Belkin and P. Niyogi. Using Manifold Structure for Partially Labelled Classification . Neural Information Processing Systems 15 (NIPS’2002), pp. 929-936, 2003.
• M. Belkin and P. Niyogi. Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering . Neural Information Processing Systems 14 (NIPS’2001), pp. 585-591, 2002.
• Xiaofei He. Laplacian Eigenmap for Image Retrieval , Master’s thesis, Computer Science Department, the University of Chicago, 2002.

Principal curves

Note: The site by K¨¦gl is probably a better resource on principal curves.

• B. K¨¦gl , A. Krzyzak. Piecewise linear skeletonization using principal curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 59-74, 2002. PDF ( Java implementation )
• B. K¨¦gl, A. Krzyzak, T. Linder, K. Zeger. Learning and design of principal curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 3, pp. 281-297, 2000. IEEE Xplore
• R. Tibshirani. Principal curves revisited. Statistics and Computing , vol 2, pp. 183–190, 1992
• T. Hastie and W. Stuetzle. Principal curves. Journal of the American Statistical Association, vol 84, pp. 502–516, 1989
• J.J. Verbeek, N. Vlassis, B. Krose A soft k-Segments Algorithm for Principal Curves . Proc. Int. Conf. on Artificial Neural Networks, 2001.
• A. Smola, R. C. Williamson, S. Mika, and B. Scholkopf. Regularized principal manifolds . In Computational Learning Theory: 4th European Conference, volume 1572 of Lecture Notes in Artificial Intelligence, pages 214 — 229. Springer, 1999.

Charting/co-ordination

• J.J. Verbeek, S.T. Roweis and N. Vlassis. Non-linear CCA and PCA by Alignment of Local Models. Neural Information Processing System 16 (NIPS’2003).
• D. De Ridder and V. Franc. Robust manifold learning. Technical report CTU-CMP-2003-08, Center for Machine Perception, Department of Cybernetics Faculty of Electrical Engineering, Czech Technical University, Prague, 2003, pp. 1-36.
• Y. W. Teh and S. Roweis. Automatic Alignment of Local Representations Neural Information Processing Systems 15 (NIPS’2002).
• M. Brand. Charting a manifold . Neural Information Processing Systems 15 (NIPS’2002)
• J.J. Verbeek, N. Vlassis, B. Krose. Coordinating Principal Component Analyzers . Proceedings of International Conference on Artificial Neural Networks , Madrid, Spain, 2002.
• S. Roweis, L. Saul and G. Hinton. Global Coordination of Local Linear Models . Neural Information Processing Systems 14 (NIPS’2001) , pp. 889–896, 2002.

Common issues

• FastMap, MetricMap, and Landmark MDS are all Nystrom Algorithms
• Y. Bengio, J-F. Paiement, and P. Vincent. Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering. NIPS 16, 2003.
• Y. Bengio and P. Vincent and J.-F. Paiement. Spectral Clustering and Kernel PCA are Learning Eigenfunctions. CIRANO Working Papers from CIRANO.
• Ham J., D. D. Lee, S. Mika and B. Schölkopf. A kernel view of the dimensionality reduction of manifolds. Technical Report TR-110, Max-Planck-Institut f¨¹r biologische Kybernetik, T¨¹bingen, July 2003.
• Matthew Brand and Kun Huang. A unifying theorem for spectral embedding and clustering. 9th International Conference on Artificial Intelligence and Statistics, Key West, Florida.
• C. K. I.Williams. On a connection between kernel PCA and metric multidimensional scaling. In Advances in Neural Information Processing Systems 13. MIT Press, 2001.

Estimating intrinsic dimensionality

• E. Levina and P.J. Bickel. Maximum Likelihood Estimation of Intrinsic Dimension. Advances in Neural Information Processing Systems 17 (NIPS2004). MIT Press, 2005.
• J. Costa and A. O. Hero. Manifold learning using Euclidean K-nearest neighbor graphs. Proceedings of IEEE International Conference on Acoustic Speech and Signal Processing, vol. 4, pp. 988-991, Montreal, May, 2004.
• J. Costa and A. O. Hero, Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Transactions on Signal Processing, vol. 25, no. 8, pp. 2210-2221, August 2004.
• B. Bal¨¢zs K¨¦gl. Intrinsic Dimension Estimation Using Packing Numbers . Neural Information Processing Systems 15 (NIPS’2002), 2003.
• F. Camastra and A. Vinciarelli. Estimating the Intrinsic Dimension of Data with a Fractal-Based Method . IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 24, no. 10, Oct 2002.
• J. Bruske and G. Sommer. Intrinsic dimensionality estimation with optimally topology preserving maps. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, number 5, pp. 572¨C575, 1998.
• P. Verveer and R. Duin. An evaluation of intrinsic dimensionality estimators. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 1, pp. 81-86, 1995.
• K. Pettis and T. Bailey and A. K. Jain and R. Dubes. An Intrinsic Dimensionality Estimator from Near-Neighbor Information. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 1, no. 1, pp. 25-36, 1979.
• K. Fukunaga and D.R. Olsen. An algorithm for finding intrinsic dimensionality of data. IEEE Transactions on Computers, C-20:176¨C183, 1971.

SOM and related

• D. K. Agrafiotis and H. Xu. A self-organizing principle for learning nonlinear manifolds
• C. M. Bishop, M. Svensen and C. K. I. Williams. GTM: the generative topographic mapping. Neural Computation, vol 10, pp. 215–234, 1998
• J.J. Verbeek, N. Vlassis, B. Krose. The Generative Self-Organizing Map: a probabilistic generalization of Kohonen’s SOM

Miscellaneous methods

• Proximity Graphs for Clustering and Manifold Learning. Miguel A. Carreira-Perpinan, Richard S. Zemel. NIPS 2004.
• Non-Local Manifold Tangent Learning. Yoshua Bengio, Martin Monperrus. NIPS 2004.
• Adaptive Manifold Learning. Jing Wang, Zhenyue Zhang, Hongyuan Zha. NIPS 2004.
• Distributional Scaling: An Algorithm for Structure-Preserving Embedding of Metric and Nonmetric Spaces. Michael Quist, Golan Yona. Journal of Machine Learning Research. 5(Apr):399–420, 2004. [abs]
• Face Recognition from Face Motion Manifolds Using Robust Kernel Resistor-Average Distance. O. Arandjelovic and R. Cipolla. IEEE Workshop on Face Processing in Video, 2004.
• Learning with Cascade for Classification of Non-Convex Manifolds. X. Huang, S. Z. Li, Y. Wang.
• K. Q. Weinberger, F. Sha, and L. K. Saul (2004). Learning a kernel matrix for nonlinear dimensionality reduction. Proceedings of the Twenty First International Confernence on Machine Learning (ICML-04), Banff, Canada
• Kilian Q. Weinberger. Lawrence K. Saul. Unsupervised learning of image manifolds by semidefinite programming. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), 27 June – 2 July 2004, Volume 2, pages 988-995.
• Joan Glaunes. Alain Trouv¨¦. Laurent Younes. Diffeomorphic Matching of Distributions: A New Approach for Unlabelled Point-Sets and Sub-Manifolds Matching. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 04), vol.2, 712-718, 2004.
• C. Hu, Y. Chang, R. Feris,  M. Turk. Manifold Based Analysis of Facial Expression The First IEEE Workshop on Face Processing in Video. June 28, 2004, Washington, D.C., USA
• Ya Chang, Changbo Hu, Matthew Turk. Probabilistic Expression Analysis on Manifolds. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 04), vol.2, 520-527, 2004.
• Local Smoothing for Manifold Learning, JinHyeong Park, Z. Zhang, Hongyuan Zha and R. Kasturi. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 04), vol.2, 452-459, 2004.
• A. Elgammal, C.S. Lee. Separating Style and Content on a Nonlinear Manifold. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 04), Washington, DC, June 26-July 2nd, 2004.
• A. Elgammal, C.S. Lee ¡°Inferring 3D Body Pose from Silhouettes using Activity Manifold Learning¡± IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 04), Washington, DC, June 26-July 2nd, 2004. [PDF]
• A. Elgammal ¡°Nonlinear Generative Models for Dynamic Shape and Dynamic Appearance¡± 2nd International Workshop on Generative-Model based vision. GMBV 2004, Washington DC, USA, June 2004 in association with CVPR04. [PDF]
• C. Sminchisescu, A. Jepson. Generative Modeling for Continuous Non-Linearly Embedded Visual Inference. International Conference on Machine Learning, ICML 2004.
• D. L. Donoho and C. Grimes. Hessian Eigenmaps: new locally linear embedding techniques for high-dimensional data . Technical Report TR-2003-08, Department of Statistics, Stanford University, 2003.
• D. De Ridder and V. Franc. Robust manifold learning. Technical report CTU-CMP-2003-08, Center for Machine Perception, Department of Cybernetics Faculty of Electrical Engineering, Czech Technical University, Prague, 2003.
• G. Hinton and S. Roweis. Stochastic Neighbor Embedding . Advances in Neural Information Processing Systems 14 (NIPS’2001).
• D. Freedman. Efficient simplicial reconstructions of manifolds from their samples . IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10):1349 -1357, 2002.
• Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment . Zhenyue Zhang and Hongyuan Zha. CSE-02-019, Technical Report, CSE, Penn State Univ., 2002.
• A Unified Model for Probabilistic Principal Surfaces. K. Chang and J. Ghosh. IEEE Trans. PAMI, 23(1), Jan 2001, pp. 22-41.
• N. Kambhatla and T. K. Lee. Dimension reduction by local principal component analysis. Neural Computation, 9:1493 1516, (1997)
• G. E. Hinton and P. Dayan and M. Revow. Modeling the manifolds of handwritten digits, IEEE Transactions on Neural Networks, 8(1), Jan 1997, pp. 65-74.
• D. Zhou, J. Weston, A. Gretton, O. Bousquet and B. Schölkopf. Ranking on Data Manifolds. NIPS16, 2003.
• D. Zhou, B. Schölkopf and T. Hofmann: Semi-supervised Learning on Directed Graphs. Advances in Neural Information Processing Systems 17 (2005)
• D. Zhou and B. Schölkopf: A Regularization Framework for Learning from Graph Data. Workshop on Statistical Relational Learning at Twenty-first International Conference on Machine Learning (2004)
• D. Zhou and B. Schölkopf: Learning from Labeled and Unlabeled Data Using Random Walks. DAGM’04: 26th Pattern Recognition Symposium (2004)
• D. Zhou, O. Bousquet, T.N. Lal, J. Weston and B. Schölkopf: Learning with Local and Global Consistency. Advances in Neural Information Processing Systems 16, 321-328. (Eds.) Thrun, S., L. Saul and B. Schölkopf, MIT Press, Cambridge, Mass. (2004)
• N. Vlassis, Y. Motomura, and B. Krose. Supervised dimension reduction of intrinsically low-dimensional data. Neural Computation, 14(1), January 2002.
• X. Zhu, Z. Ghahramani and J. Lafferty. Semi-supervised Learning Using Gaussian Fields and Harmonic Functions. ICML 2003.
• S. Schaal and S. Vijayakumar and C. Atkeson. Local Dimensionality Reduction. Advances in Neural Information Processing Systems 10, pp.633-639, 1998.
• S. Vijayakumar and S. Schaal. Local Dimensionality Reduction for Locally Weighted Learning. Proc. IEEE International Symposium on Computational Intelligence in Robotics and Automation, (CIRA’97), pp.220-225, 1997.

Others

• C.J.C. Burges, “Geometric Methods for Feature Extraction and Dimensional Reduction“, to appear in “Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers”, Eds. L. Rokach and O. Maimon, Kluwer Academic Publishers, 2005, 36 pages
• J. Zhang, S. Z. Li, and J. Wang. “Manifold Learning and Applications in Recognition“. in Intelligent Multimedia Processing with Soft Computing. Springer-Verlag, Heidelberg. 2004.
• Diffusion geometry
• Optimal Manifold Representation of Data: An Information Theoretic Approach. NIPS 2003. Denis V. Chigirev, William S. Bialek [ps.gz][pdf]
• A. Shashua, A. Levin and S. Avidan. Manifold Pursuit: A New Approach to Appearance Based Recognition. International Conference on Pattern Recognition (ICPR), August 2002, Quebec, Canada.
• Dimitris K. Agrafiotis and Huafeng Xu. A self-organizing principle for learning nonlinear manifolds . PNAS 2002 99: 15869-15872
• M. Vlachos, C. Domeniconi, D. Gunopulos, G. Kollios, N. Koudas. Non-Linear Dimensionality Reduction Techniques for Classification and Visualization , Proc. of 8th SIGKDD, Edmonton, Canada, 2002, pp. 645-651
• C.Bregler, S.Omohundro, Nonlinear Image Interpolation using Manifold Learning, in Advances in Neural Information Processing Systems 7, 1995
• C.Bregler, S.Omohundro, Surface Learning with Applications to Lipreading, in Cowan, J.D., Tesauro, G., and Alspector, J. (eds.), Advances in Neural Informantion Precessing Systems 6. San Francisco, CA: Morgan Kaufmann Publishers, 1994. (gzip-postscript)
• X. Huo and J. Chen. LOCAL LINEAR PROJECTION (LLP) . In the Proceedings of the Workshop on Genomic Signal Processing and Statistics (GENSIPS).
• R. Pless and I. Simon. Embedding Images in non-Flat Spaces . Technical Report WU-CS-01-43, Washington University.
• J.J. Verbeek, N. Vlassis and B. Krose Fast nonlinear dimensionality reduction with topology preserving networks .
• Stan Z. Li, Rong Xiao, ZeYu Li, HongJiang ZhangMulti-View Face Patterns to a Gaussian Distribution in a Low Dimensional Space.
• N. P. Hughes and D. Lowe. Artefactual Structure from Least-squares Multidimensional Scaling . Neural Information Processing Systems 15 (NIPS’2002)
• D. J. Navarro and M. D. Lee. Combining Dimensions and Features in Similarity-Based Representations . Neural Information Processing Systems 15 (NIPS’2002)
• J. Weston, O. Chapelle, A. Elisseeff, B. Schoelkopf, and V. Vapnik. Kernel Dependency Estimation . Neural Information Processing Systems 15 (NIPS’2002)
• Carreira-Perpinan. A review of Dimension Reduction Techniques . Technical Report, Department of Computer Science, University of Sheffield, 1997.
• local tangent space embedding
• Fractal dimension and Dimensionality Reduction
• A Dimensionality Reduction Approach to Modeling Protein Flexibility
• SVM Decision Boundary Based Discriminative Subspace Induction
• Kernel Methods for Computer Vision: Theory and Applications
• Learning from Labeled and Unlabeled Data using Graph Mincuts

Applications

• M. Niskanen and O. Silv¨¦n. Comparison of dimensionality reduction methods for wood surface inspection. Proc. 6th International Conference on Quality Control by Artificial Vision (QCAV 2003), May 19-23, Gatlinburg, Tennessee, USA.
• Hadid A, Kouropteva O & Pietikäinen M. Unsupervised learning using locally linear embedding: experiments in face pose analysis. Proc. 16th International Conference on Pattern Recognition, August 11-15, Quebec City, Canada, 1:111-114.
• O. C. Jenkins and M. J Mataric’. Deriving Action and Behavior Primitives from Human Motion Data . In the IEEE/RSJ Internation Conference on Intelligent Robots and Systems (IROS-2002), pages 2551-2556, Lausanne, Switzerland, 2002
• A. Brun, H.-J. Park, H. Knutsson and C.-F. Westin. Coloring of DT-MRI Fiber Traces using Laplacian Eigenmaps (Extended abstract) . Eurocast 2003, Neuro Image Workshop, Las Palmas, February 2003. SPL Technical Report #369, posted May 2003.
• Image Similarity Based Image Analysis (An application of manfiold learning techniques to arrange images) WuMap project
• D. Kulpinski. LLE and Isomap Analysis of Spectra and Color Images . Master Thesis, School of Computer Science, Simon Fraser University, March 2002.
• M.-H. Yang. Face Recognition Using Extended Isomap . In ICIP 2002, 2002.
• Manifold Learning Continuous Human Motion . A web page by Huang Fei .
• C. Bregler and S. M. Omohundro. Nonlinear manifold learning for visual speech recognition . Proc. of 5th International Conference on Computer Vision, pages 494– 499, 1995.

Presentation

Lecture on manifold learning by Roweis

http://www-leibniz.imag.fr/JournApprenSlides/HighDim.pdf

“Isomap: a global geometric framework for nonlinear dimensionality reduction” by Vin de Silva

NIPS workshop talk by Carrie Grimes

ISOMAP & Image articulation by Carrie Grimes

Global Geometric Framework for Nonlinear Dimensionality Reduction: The Isomap and Locally Linear Embedding Algorithm , presented by Kristin Branson

Computer examples (of ISOMAP)

Workshop of spectral methods in dimensionality reduction, clustering, and classification in NIPS 2002

Workshop website

• Generative models implicit in spectral methods for manifold learning, by Vin de Silva and Joshua B. tenenbaum
• Mathematical Foundations for Learning Image Manifolds Using ISOMAP and LLE, by Carrie Grimes and David Donoho (related url )
• The role of the Laplace-Beltrami Operator in Learning on Manifolds, by Mikhail Belkin and Partha Niyogi
• Charting a manifold, by Matthew Brand
• Automatic Alignment of Hidden Representations, by Yee Whye Teh and Sam T. Roweis
• Convex Invariance Learning , by Tony Jebara
• Laplacians, Spectra, and Kernels, by John Lafferty
• Regularization for Continuous Data and Graphs, by Alex Smola
• Generative Models of Affinity Matrices, by Romer Rosales and Brendan Frey
• How Many Clusters? The Markov Random Walk Perspective, by Marina Meila
• Also, Schoelkopf gave an improvised talk during the kernel workshop next day on the relationship between LLE and kernel PCA

Software

Some MDS matlab code

MDS site

http://www.math.umn.edu/~wittman/mani/
Laplacian eigenmap software

VisuMap, a visualizer for high dimensional data (technical info) (paper)

Other resource page

Penn Dimensionality Reduction Reading Group

Dimensionality Reduction

Manifold Learning

Partiview & Ndaona: A good way to study Machine Learning

Partiview is a piece of software for interactive 4D-viewer dataset. It is also useful for 3D visualization of dataset in Machine Learning algorithm, i.e. show manifolds or classification results.

Ndaona is a piece of Matlab functions for Partiview-compatible data conversion.

Downloads

• PartiView can be reached here (binaries + source codes)
• Ndaona is here

Screenshots

I borrow some screenshots from Ndaona’s website to illustrate here:

How well a linear svm (in LIBSVM) classifies Mandarin tones

How well does Principal Components Analysis separate faces

Resources on Complex Networks

• Exploring complex networks http://www.nature.com/nature/journal/v410/n6825/full/410268a0.html#Complex-network-architectures
• The New Science of Networks http://pages.uoregon.edu/vburris/hc431/
• Linked: The New Science of Networks http://www.amazon.com/Linked-Science-Networks-Albert-L%C3%A1szl%C3%B3-Barab%C3%A1si/dp/0738206679
• Resources in Complex Networks http://cyvision.ifsc.usp.br/networks/
• Complex Networks notes http://cscs.umich.edu/~crshalizi/notabene/complex-networks.html