NLP: Supervised Sequence Prediction (Text to SQL)

project

May 2024

Translating Natural Language Instructions to SQL Queries (T5 Model)

This assignment focuses on supervised sequence prediction, specifically translating natural language instructions into SQL queries. Three different approaches will be explored: fine-tuning a pre-trained encoder-decoder transformer model (T5), training a similar model from scratch, and using diverse prompt engineering techniques with a large language model (LLM).

Tasks include:

1. Fine-tuning a pre-trained T5 model.

2. Training a T5 model from scratch.

3. Using in-context learning with instruction-tuned LLMs (Gemma 1.1 2B and CodeGemma 7B).

Evaluation will be based on whether the generated SQL queries return the same database records as the ground-truth using the F1 metric. The assignment also includes a milestone task with an F1 score requirement of at least 0.5 on the test set by May 7.

Resources provided include a starter repository, data sets, and utility functions for evaluation. The assignment encourages experimentation with data processing, tokenization, and prompt design to improve model performance.

Submission involves code in a GitHub repository and a report following the provided template. The leaderboard ranks submissions based on F1 scores, with the final grade considering empirical performance, methodology, and code quality.

This assignment aims to provide experience in sequence prediction tasks, emphasizing empirical work and thorough analysis.

Sequence Prediction
SQL
Natural Language Processing
T5 Model

Source Code

Whiteboard