NLP: Supervised Sequence Prediction (Text to SQL)

May 2024

Translating Natural Language Instructions to SQL Queries (T5 Model)

  • This assignment focuses on supervised sequence prediction, specifically translating natural language instructions into SQL queries. Three different approaches will be explored: fine-tuning a pre-trained encoder-decoder transformer model (T5), training a similar model from scratch, and using diverse prompt engineering techniques with a large language model (LLM).
  • Tasks include:
  • 1. Fine-tuning a pre-trained T5 model.
  • 2. Training a T5 model from scratch.
  • 3. Using in-context learning with instruction-tuned LLMs (Gemma 1.1 2B and CodeGemma 7B).
  • Evaluation will be based on whether the generated SQL queries return the same database records as the ground-truth using the F1 metric. The assignment also includes a milestone task with an F1 score requirement of at least 0.5 on the test set by May 7.
  • Resources provided include a starter repository, data sets, and utility functions for evaluation. The assignment encourages experimentation with data processing, tokenization, and prompt design to improve model performance.
  • Submission involves code in a GitHub repository and a report following the provided template. The leaderboard ranks submissions based on F1 scores, with the final grade considering empirical performance, methodology, and code quality.
  • This assignment aims to provide experience in sequence prediction tasks, emphasizing empirical work and thorough analysis.
    • Sequence Prediction
    • SQL
    • Natural Language Processing
    • T5 Model
    We use cookies and similar technologies to provide certain features, enhance the user experience and deliver content that is relevant to your interests. Depending on their purpose, analysis and marketing cookies may be used in addition to technically necessary cookies. By clicking on "Agree and continue", you declare your consent to the use of the aforementioned cookies. you can make detailed settings or revoke your consent (in part if necessary) with effect for the future. For further information, please refer to ourPrivacy Policy.

    Early to bed and early to rise, makes a man healthy, wealthy, and wise. ——Benjamin Franklin

    © 2023 Yuyang Wang's Site. Powered by React.js