Parameter tying encourages the similarities between parameters, and this is typically done by adding regularization. For example, any differences in parameters pairwise could be punished.

University of Notre Dame

There are two strategies for constraining parameters of one task by other tasks:

1)	Parameter Sharing

2)	Parameter Tying



Strategy of Multitask Learning

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Generalizing from a Few Examples: A Survey on Few-Shot Learning

Parameter sharing directly shares parameters across different tasks and models to promote the training of few-shot tasks.

Learn Before

Related