Pro DNAnexus Workspace

A scientific computing skill for working with DNAnexus — the cloud-based genomics platform for managing, analyzing, and sharing large-scale genomic data. Pro DNAnexus Workspace helps you upload data, run bioinformatics workflows, manage projects, and access results through the DNAnexus API and command-line tools.

When to Use This Skill

Choose Pro DNAnexus Workspace when:

• Running bioinformatics pipelines on DNAnexus cloud infrastructure
• Managing large genomic datasets (WGS, WES, RNA-seq) in DNAnexus projects
• Building and executing DNAnexus applets and workflows
• Accessing UK Biobank or other large cohort data hosted on DNAnexus

Consider alternatives when:

• You want to run pipelines locally (use Nextflow or Snakemake)
• You need AWS-native genomics (use Amazon Omics)
• You need a different cloud genomics platform (use Terra/FireCloud)
• You're doing small-scale analysis (local tools are simpler)

Quick Start

Copy
claude "Upload FASTQ files to DNAnexus and run a variant calling pipeline"

Copy
# Install and configure DX CLI
pip install dxpy
dx login

# Create a project
dx new project "My Genomics Project" --region aws:us-east-1

# Select the project
dx select "My Genomics Project"

# Upload FASTQ files
dx upload sample_R1.fastq.gz sample_R2.fastq.gz --destination /raw_data/

# List available apps
dx find apps --name "BWA-MEM"

# Run alignment pipeline
dx run app-bwa-mem \
  -ireads_fastqgz=raw_data/sample_R1.fastq.gz \
  -ireads2_fastqgz=raw_data/sample_R2.fastq.gz \
  -igenome_reference=project-xxxx:file-xxxx \
  --destination /aligned/

# Monitor job
dx watch job-xxxx

Core Concepts

DNAnexus Object Types

Object	Description	CLI Command
Project	Container for data and analyses	dx new project
File	Data file (FASTQ, BAM, VCF)	dx upload, dx download
App	Packaged analysis tool	dx run app-name
Applet	Custom analysis script	dx build, dx run
Workflow	Multi-step pipeline	dx run workflow-name
Job	Running analysis instance	dx describe job-xxx

Running Pipelines

Copy
# Run GATK HaplotypeCaller
dx run app-gatk4-haplotypecaller \
  -ibam=aligned/sample.bam \
  -iref_genome=reference/hg38.fa \
  -iintervals=targets/exome.bed \
  --destination /variants/ \
  --instance-type mem2_ssd1_v2_x8

# Run a workflow (multi-step)
dx run workflow-exome-pipeline \
  -istage-1.reads=raw_data/sample_R1.fastq.gz \
  -istage-1.reads2=raw_data/sample_R2.fastq.gz \
  --destination /results/

Python SDK

Copy
import dxpy

# Find files in a project
files = list(dxpy.find_data_objects(
    classname="file",
    name="*.vcf.gz",
    name_mode="glob",
    project=dxpy.PROJECT_CONTEXT_ID
))

for f in files:
    desc = dxpy.describe(f["id"])
    print(f"{desc['name']} | {desc['size'] / 1e9:.1f} GB | {desc['state']}")

# Download results
dxpy.download_dxfile("file-xxxx", "local_output.vcf.gz")

# Run an app programmatically
job = dxpy.DXApp(name="bwa-mem").run({
    "reads_fastqgz": dxpy.dxlink("file-xxxx"),
    "reads2_fastqgz": dxpy.dxlink("file-yyyy"),
    "genome_reference": dxpy.dxlink("file-zzzz")
})
job.wait_on_done()

Configuration

Parameter	Description	Default
region	Cloud region for project	aws:us-east-1
instance_type	Compute instance for jobs	App-specific default
priority	Job priority (normal, high)	normal
billing_account	Cost allocation account	Default org account
output_folder	Default output directory	/

Best Practices

1. Choose the right instance type. Under-provisioned instances cause failures; over-provisioned ones waste money. Use mem2_ssd1_v2_x8 for alignment, mem3_ssd1_v2_x16 for variant calling with large BAM files. Check app documentation for recommended instances.

2. Organize projects with consistent folder structure. Use /raw_data/, /aligned/, /variants/, /qc/ conventions. Consistent organization makes it easy to find data months later and enables automated pipelines that reference standard paths.

3. Use batch processing for multiple samples. Instead of launching jobs one at a time, use dx run --batch-tsv with a TSV file listing all samples. This is faster to set up and easier to monitor than individual job launches.

4. Monitor costs during execution. Large genomic analyses can be expensive. Use dx describe job-xxx --json | jq .totalPrice to check running costs. Set project spending limits to prevent unexpected bills from misconfigured jobs.

5. Download only what you need. Full WGS datasets can be terabytes. Download specific files (VCF, QC reports) rather than entire project contents. Use streaming access (dx cat) for quick file inspection without downloading.

Common Issues

Job fails with "insufficient resources." The instance type doesn't have enough memory or storage for the data size. Scale up the instance: use --instance-type mem3_ssd1_v2_x32 for memory-intensive tasks. Check job logs for the specific resource that was exhausted.

File upload stalls on large files. Use dx upload --nthreads 8 for parallel upload streams. For very large files (>100GB), use dx-toolkit's chunked upload. Ensure stable network connectivity — interrupted uploads must restart from the beginning.

App version mismatch produces different results. Pin app versions in your commands: dx run app-bwa-mem/1.2.0 instead of app-bwa-mem (latest). Version pinning ensures reproducible results across time and team members.