SQLForge-7B

A fine-tuned Qwen2.5-7B-Instruct model specialized for natural language to SQL generation. Given a database schema and a question in plain English, it writes the correct SQL query and explains what it does.

Key Details

Dataset

Trained on 10,000 examples from the gretelai/synthetic_text_to_sql dataset, which covers 100 domains with a wide range of SQL complexity levels including subqueries, joins, aggregations, window functions, and set operations. Each example includes the database schema (CREATE TABLE statements), a natural language question, the correct SQL query, and an explanation.

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("sriksven/SQLForge-7B")
tokenizer = AutoTokenizer.from_pretrained("sriksven/SQLForge-7B")

messages = [
    {
        "role": "system",
        "content": "You are an expert SQL assistant. Given a database schema and a natural language question, write the correct SQL query and explain what it does.",
    },
    {
        "role": "user",
        "content": (
            "Schema:\n"
            "CREATE TABLE employees (id INT, name VARCHAR(100), department VARCHAR(50), salary DECIMAL(10,2));\n"
            "CREATE TABLE departments (name VARCHAR(50), budget DECIMAL(12,2));\n\n"
            "Question: What is the average salary by department, only showing departments with average salary above 75000?"
        ),
    },
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Unsloth (faster inference)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="sriksven/SQLForge-7B",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

SQL Complexity Coverage

The training data includes queries across multiple complexity levels:

• Simple SELECT with WHERE clauses
• Aggregations with GROUP BY and HAVING
• Single and multiple JOINs
• Subqueries and correlated subqueries
• Window functions (ROW_NUMBER, RANK, LAG, LEAD)
• Set operations (UNION, INTERSECT, EXCEPT)
• Data definition (CREATE, ALTER, INSERT)

Intended Use

• Natural language interfaces to databases
• SQL copilot tools for analysts and developers
• Educational tools for learning SQL
• Prototyping data query systems

Limitations

• Trained on synthetic data, not real production database queries
• May not handle highly domain-specific or proprietary SQL dialects
• Best with standard SQL syntax (PostgreSQL/MySQL style)
• Does not validate against a live database — SQL correctness is not guaranteed
• Long or deeply nested schemas may exceed the 2048 token context

Training Infrastructure

Source Code

Training scripts: github.com/sriksven/LLM-FineTune-Suite

License

Apache 2.0