pedro castanheiraPT
← All work

Production · education company

Assignment grading at scale

Designed an asynchronous NestJS pipeline on BullMQ that syncs submissions from the LMS, sends each one to LangChain grading and writes the grade and feedback back to Moodle.

Role
Tech lead · architecture and development
Status
In production
Stack
  • TypeScript
  • NestJS
  • BullMQ
  • Redis
  • PostgreSQL
  • MongoDB
  • LangChain
  • Moodle
  • GCS
  • WebSocket
cron / manual~140k submissions/monthLMS(Moodle)activityjobsyncsjobs persubmission3 retries · backoff· dedupe · re-queuePython serviceLangChaingrade +feedbackstatus +resultevents +durationsWebSocket~140k submissions/month~99% lower cost per assignment vs. human grading (estimate)estimated ROI of ~3,000%

Problem

Grade assignments at volume, with consistent criteria, inside the LMS students and teachers already use — without losing submissions when an external service fails or slows down.

Available data

Submissions arrive from the LMS as a file or online text, with activity, offering and organization metadata. Files live in Google Cloud Storage.

Architecture decisions

A staged pipeline on queues

One job per activity syncs the submissions and creates one job per submission. Each stage — file, analysis, integration — runs on BullMQ workers, outside the request.

Fail without losing work

Analysis jobs retry up to three times with exponential backoff, deduplicate per answer and keep a per-submission state, so stuck work can be re-queued.

AI in a separate service

NestJS orchestrates; LangChain grading runs in its own Python service over HTTP. One side scales the queue; the other evolves the model and the criteria.

Viability measured against the real alternative

Cost per graded assignment was compared with human grading, which is what the product replaces.

Implementation

  • Cron or manual trigger → activity job → submission sync with the LMS.
  • BullMQ jobs per submission: file download, call to the grading service and result persistence.
  • Grade and feedback written back to Moodle.
  • Events and durations of each run stored for operational dashboards; progress pushed over WebSocket.

Result

  • ~140ksubmissions graded per month
  • ~99%lower cost per graded assignment vs. human grading · estimate
  • ~3,000%ROI · estimate

Cost reduction and ROI are internal estimates compared with human grading.

Limits and lessons

  • The grading itself is a simple LangChain chain; the hard part is orchestration, resilience and LMS integration.
  • Cost reduction and ROI are estimates, not audited figures; I do not detail the formula or the period here.
  • Grading criteria and student data stay out for confidentiality.
Next caseDocument triage before OCR →