Lifelong Machine Learning
Introduction
Definition
Key Characteristics
- Continuous learning process
- Knowledge accumulation in KB
- Use of past knowledge to help future learning
Components of LML
1. Knowledge Base (KB)
- Past Information Store (PIS)
- Knowledge Store (KS)
- Knowledge Miner (KM)
- Knowledge Reasoner (KR)
2. Knowledge-Based Learner (KBL)
Other Related Learning Methods and Differences with LML
Transfer Learning—Finding correspondences(Pivot features, non-pivot features):
Transfer learning typically
uses labeled training data from one (or more)
source domain(s) to help learning in the target domain
that has little or no labeled data (Aue and
Gamon, 2005, Bollegala et al., 2011). It does not
use the results of the past learning or knowledge
mined from the results of the past learning. Further,
transfer learning is usually inferior to traditional
supervised learning when the target domain
already has good training data.
Multi-task Learning—Each task model is a linear combination of shared latent components, All task models share latent basic model components
Online Learning—
The Differences of methods above with LML
the Online learning:
Lifelong Supervised Learning
Early Works
1. Memory based + Distance calculate predicting
2. Neural Network
3. Task Clustering
Nowadays Works
1. ELLA: based on GO-MTL
N: all tasks (individual)
L: all tasks learned from N (latent)
2. Lifelong Sentiment Classification
Components
i. A classifier for sentiment classification: Use BN and stochastic gradient descent.
ii. Embed the past knowledge by using penalty terms so that optimizing the method.
iii. Also, consider about dependent classes.
3. Cumulative Learning
Goal: Detecting unseen classes in testing.
Steps:
i. Add new classes
ii. Self Learning