Controllable Text Generation Using Pre-Trained Language Models
Many controllable text generation methods use autoregressive or sequence-to-sequence pre-trained language models to generate text under a task-specific control condition. Given a control condition and output sequence , generation is modeled as . In an autoregressive model, this distribution is factorized as P(Ymid C)=prod_{n=1}^{N}p(y_nmid y_{1:n-1},C), so each token depends on both the preceding tokens and the control condition.
0
1
Contributors are:
Who are from:
Tags
Deep Learning (in Machine learning)
Data Science
Computing Sciences
Related
Representative Transformer-based PLMs
Analysis of Language Model Training Strategies
A startup is developing a system to classify medical research abstracts into different fields of study (e.g., cardiology, oncology, neurology). They have a limited dataset of 10,000 labeled abstracts. Which of the following statements best justifies the decision to use a large, pre-trained language model and fine-tune it, rather than training a new model from scratch on their dataset?
A development team is building a system to classify news articles into categories like 'Sports', 'Technology', and 'Politics'. They are using a modern approach that starts with a large, general-purpose language model. Arrange the following stages of their development process into the correct chronological order.
Traditional Role of Language Models
LLMs as Complete Systems in Generative AI