Advanced ML Paper Writing

Overview

Writing publication-ready machine learning papers for top-tier conferences (NeurIPS, ICML, ICLR, ACL, AAAI, COLM) requires a specific set of skills that go beyond technical expertise. This template provides expert-level guidance synthesized from leading ML researchers including Neel Nanda (Google DeepMind), Sebastian Farquhar (DeepMind), Gopen and Swan (scientific writing principles), Zachary Lipton, Jacob Steinhardt (UC Berkeley), Ethan Perez (Anthropic), and Andrej Karpathy. The core philosophy is that a paper is a short, rigorous, evidence-based technical story with a single clear contribution that readers care about. This template covers the complete paper writing workflow: starting from a research repository, defining the narrative, structuring sections, writing with clarity, managing citations without hallucination, formatting for specific conferences, and navigating the review process. Every aspect is oriented toward maximizing acceptance probability at top venues.

When to Use

• Starting a paper from research code: Transform experimental results and code into a structured paper narrative.
• Drafting or revising sections: Get guidance on writing each section (abstract, intro, methods, experiments, related work).
• Finding and verifying citations: Use programmatic citation workflows to avoid hallucinated references.
• Formatting for conference submission: Apply correct templates, page limits, and venue-specific requirements.
• Resubmitting to a different venue: Convert format and adjust content for a new conference.
• Responding to reviewer feedback: Address reviewer concerns in rebuttals and revised submissions.

Quick Start

Repository-First Workflow

Copy
# 1. Explore the research repository
ls -la
find . -name "*.py" | head -20
find . -name "*.md" -o -name "*.txt" | xargs grep -l -i "result\|conclusion"

# 2. Find existing citations
grep -r "arxiv\|doi\|cite" --include="*.md" --include="*.bib" --include="*.py"
find . -name "*.bib"

# 3. Set up paper directory from template
cp -r templates/neurips2025/ ~/papers/my-paper/
cd ~/papers/my-paper/

# 4. Verify template compiles
latexmk -pdf main.tex

The One-Sentence Test

Before writing anything, answer this: What is the single thing your paper contributes?

If you cannot state it in one sentence, you do not yet have a paper. Examples:

• "We prove that sparse autoencoders recover 70% of interpretable features in GPT-2."
• "We demonstrate that chain-of-thought prompting improves reasoning by 40% on GSM8K."
• "We introduce a method that reduces LLM inference latency by 3x without quality loss."

Core Concepts

The Narrative Principle

Every successful ML paper centers on a narrative with three pillars:

Pillar	Description	Example
The What	1-3 specific novel claims	"We prove X achieves Y under condition Z"
The Why	Rigorous evidence supporting claims	Strong baselines, distinguishing experiments
The So What	Why readers should care	Connection to recognized community problems

The Five-Sentence Abstract (Farquhar Formula)

Copy
\begin{abstract}
% Sentence 1: What you achieved
We introduce MethodX, a technique for reducing transformer inference latency
by 3x without quality degradation.
% Sentence 2: Why this is hard and important
Efficient inference is critical for deploying LLMs at scale, yet existing
approaches sacrifice quality for speed.
% Sentence 3: How you do it (with keywords for discoverability)
MethodX uses speculative decoding with learned draft models optimized via
knowledge distillation from the target model.
% Sentence 4: What evidence you have
We evaluate on 5 benchmarks across 3 model families, showing consistent
speedups with <1\% quality loss.
% Sentence 5: Your most remarkable result
On Llama-3-70B, MethodX achieves 3.2x speedup while maintaining 99.4\%
of the original MMLU accuracy.
\end{abstract}

Section-by-Section Writing Guide

Introduction (1-1.5 pages max)

Copy
% Paragraph 1: Problem and motivation
% Paragraph 2: Why existing approaches fall short
% Paragraph 3: Your approach (high level)
% Paragraph 4: Contribution list
\paragraph{Contributions.} We make the following contributions:
\begin{itemize}[leftmargin=*,nosep]
\item We propose MethodX, which...
\item We prove that MethodX achieves...
\item We demonstrate on N benchmarks that...
\end{itemize}
% Methods should start by page 2-3

Methods Section

Copy
% Enable reimplementation:
% - Conceptual outline or pseudocode
% - All hyperparameters listed
% - Architectural details sufficient for reproduction
% Present final design decisions; ablations go in experiments

\begin{algorithm}
\caption{MethodX Training Procedure}
\begin{algorithmic}[1]
\STATE Initialize draft model $M_d$ from target model $M_t$
\FOR{each training step}
    \STATE Sample batch $B$ from training data
    \STATE Compute target logits: $z_t = M_t(B)$
    \STATE Compute draft logits: $z_d = M_d(B)$
    \STATE Update $M_d$ to minimize $\text{KL}(z_t \| z_d)$
\ENDFOR
\end{algorithmic}
\end{algorithm}

Experiments Section

Copy
% For each experiment, state:
% 1. What claim it supports
% 2. How it connects to main contribution
% 3. Experimental setting
% 4. What to observe

\subsection{Main Results}
% "This experiment tests whether MethodX achieves comparable
% quality to the baseline while providing speedup."

\begin{table}[t]
\centering
\caption{Main results on standard benchmarks. MethodX achieves
3.2x average speedup with minimal quality loss. Bold indicates
best result; ↑ means higher is better.}
\begin{tabular}{lcccc}
\toprule
Method & MMLU ↑ & GSM8K ↑ & HumanEval ↑ & Speedup ↑ \\
\midrule
Baseline & 79.2 & 85.1 & 72.0 & 1.0x \\
PriorWork & 78.5 & 83.2 & 70.1 & 2.1x \\
\textbf{MethodX} & \textbf{78.8} & \textbf{84.7} & \textbf{71.5} & \textbf{3.2x} \\
\bottomrule
\end{tabular}
\end{table}

Citation Verification Workflow

Copy
# CRITICAL: Never write BibTeX from memory. Always fetch programmatically.

import requests

def doi_to_bibtex(doi: str) -> str:
    """Get verified BibTeX from DOI via CrossRef."""
    response = requests.get(
        f"https://doi.org/{doi}",
        headers={"Accept": "application/x-bibtex"}
    )
    response.raise_for_status()
    return response.text

def search_semantic_scholar(query: str, limit: int = 5):
    """Search for papers via Semantic Scholar API."""
    response = requests.get(
        "https://api.semanticscholar.org/graph/v1/paper/search",
        params={"query": query, "limit": limit,
                "fields": "title,authors,year,externalIds"}
    )
    return response.json().get("data", [])

# Workflow:
# 1. Search for the paper
results = search_semantic_scholar("attention is all you need transformers")
# 2. Verify it exists
paper = results[0]
doi = paper["externalIds"].get("DOI")
# 3. Fetch BibTeX via DOI
if doi:
    bibtex = doi_to_bibtex(doi)
else:
    print("[PLACEHOLDER - VERIFY] Could not find DOI")

Writing Style Rules

Principle	Bad	Good
Subject-verb proximity	"The model, which was trained on ImageNet using ResNet-50 with data augmentation, achieves"	"The model achieves 92% accuracy after training on ImageNet"
Stress position (emphasis at end)	"Accuracy improves by 15% when using attention"	"When using attention, accuracy improves by 15%"
Action in verbs	"We performed an analysis of"	"We analyzed"
Eliminate hedging	"This may possibly suggest"	"This demonstrates"
Consistent terminology	"method" / "approach" / "technique" (mixing)	Pick one term and use it throughout
Specific language	"good performance"	"92.1% accuracy on MMLU"
No filler words	"actually, basically, very, quite, essentially"	Delete them all

Configuration Reference

Conference Requirements

Conference	Pages	Extra (Camera-Ready)	Key Requirement
NeurIPS 2025	9	+0	Mandatory checklist
ICML 2026	8	+1	Broader Impact Statement
ICLR 2026	9	+1	LLM disclosure required
ACL 2025	8 (long)	varies	Limitations section mandatory
AAAI 2026	7	+1	Strict style adherence
COLM 2025	9	+1	Language model focus

Time Allocation (Nanda)

Section	% of Time	Rationale
Abstract	25%	100% of reviewers read it
Introduction	25%	First impression, contribution framing
Figures	25%	Many reviewers skip directly to figures
Everything else	25%	Methods, experiments, related work

Reviewer Scoring (NeurIPS Scale)

Score	Meaning	What It Takes
6	Strong Accept	Groundbreaking, flawless execution
5	Accept	Technically solid, high impact
4	Borderline Accept	Solid but limited evaluation
3	Borderline Reject	Weaknesses outweigh strengths
2	Reject	Technical flaws
1	Strong Reject	Known results or ethics issues

Best Practices

1. Define the one-sentence contribution before writing anything: If you cannot articulate what is new and why it matters in a single sentence, the paper is not ready. This sentence becomes the seed for your abstract, introduction, and title.

2. Never hallucinate citations: Use programmatic DOI-to-BibTeX conversion for every reference. If you cannot verify a citation exists, mark it as [CITATION NEEDED] and inform the human researcher. Hallucinated citations can result in desk rejection.

3. Spend equal time on abstract, intro, figures, and everything else: Most reviewers form their opinion before reaching your methods. A brilliant methods section cannot rescue a confusing abstract and introduction.

4. Delete the first sentence of your abstract if it is generic: "Large language models have achieved remarkable success in natural language processing" could start any ML paper. Start with your specific contribution instead.

5. Write experiments section with explicit claims: Begin each experiment subsection with "This experiment tests whether [specific claim]..." This forces clarity and ensures every experiment connects to your narrative.

6. Use booktabs tables with direction arrows: Bold best values, include direction symbols, right-align numbers, and maintain consistent decimal precision. Reviewers notice sloppy tables.

7. Include a limitations section: All major conferences require this. Counter-intuitively, honest acknowledgment of limitations helps because reviewers are instructed not to penalize honesty and it pre-empts their criticisms.

8. Organize related work methodologically, not paper-by-paper: "One line of work uses approach A [refs] while we use approach B because..." is far better than "Smith et al. did X. Jones et al. did Y. We do Z."

9. Use vector graphics for all plots and diagrams: PDF or EPS format for plots. Raster images (PNG) only for photographs. Use colorblind-safe palettes and verify grayscale readability.

10. Always include error bars with methodology: Specify whether you report standard deviation or standard error, the number of runs, and the random seeds used. Missing error bars are a common reviewer complaint.

Troubleshooting

Abstract too generic or long Apply the five-sentence formula strictly. Delete any sentence that could be prepended to a different ML paper. Cut to exactly 5 sentences if the conference allows, or expand to a short paragraph if not.

Introduction exceeds 1.5 pages Move background material to Related Work. Front-load the contribution bullets. The methods section should start by page 2-3 at the latest.

Reviewers say paper is hard to follow Add explicit signposting ("In this section, we show X"). Use consistent terminology (never alternate between synonyms for the same concept). Ensure every figure caption is self-contained.

Cannot find a citation programmatically Mark it as [PLACEHOLDER - VERIFY] in the LaTeX. Add a comment: % TODO: Scientist must verify this citation exists. Never invent a plausible-sounding reference.

Paper rejected from one venue, resubmitting to another Copy content (not preamble) into the new template. Adjust page limits. Address reviewer concerns without referencing the previous submission. Update references with papers published since the last submission.