392271 ISY-Project: Identify patterns of degradation in poetry generation with progressively smaller models (Pj) (SoSe 2023)

Outlook of the project:
In this project, we generate poetry with the latest large language models and see whether they outperform smaller models and by how much. We also aim for building smaller efficient models that do not degrade in poetry quality. Efficiency is a core aspect of modern machine learning in the face of increasing inequality among researchers and environmental costs of computation-heavy models. Aspects are:
1. Data collection and exploration: experiment with large language models such as ChatGPT and GPT-3-davinci for automatic poetry generation, then repeat with much smaller models. What differences are there?
2. Model building: try to build an efficient text generation model for poetry generation that does not degrade in quality.

In case the proposal would not attract enough students for a team project, it can be adapted into an individual project or a project for two students (tandem project).

Requirements for participation, required level

Required skills:
- Familiarity with deep learning framework, pytorch or tensorflow required. Basic NLP knowledge.