Turning scanned textbooks into content an AI tutor can use

A textbook page can mix an explanation, an equation and a question with answer options. Extracting the visible words is useful, but it loses the distinction between those parts. I built the pipeline to return typed blocks. A heading remains a heading. A multiple-choice question has a place for its question and options. An equation can retain notation that another application can render.

The supported blocks cover headings, paragraphs, lists, equations and tables. They also include image and diagram references, multiple-choice questions, creative questions, fill-in-the-blank questions, true-or-false questions and activity boxes. I kept these distinctions in the content model because the downstream products need to know what they are receiving. They should not have to guess whether a paragraph is an explanation or an exercise every time they read it.

That was the useful transformation: a scanned page became content with a shape the product could use. It did not make every extracted answer correct by definition. Structure gives the system a predictable way to store and exchange the result. The reading quality still depends on the page and the extraction. I kept that distinction clear when designing the output and recovery behaviour.

The education platform had textbook PDFs, but its tutor and exam features needed chapters, questions and equations as usable data. A PDF is useful for a reader who wants to open a book. It is a different starting point for a feature that needs a particular question or a chapter summary. Manually entering the contents was not a workable route for adding books to the product.

I designed and built the library content system end to end. That included the upload interface, API, shared contracts, background workers and parser. The mobile app and school dashboard connected through its integration API. Other engineers owned the core backend and the school dashboard, so this was a supporting content service within a wider product. My responsibility was to give those applications a way to upload, process and use books without rebuilding the pipeline themselves.

The difficult part was not only reading the page. AI calls can reach a quota or time out. A worker can stop while a book is being processed. A page can contain notation, tables or questions that do not fit ordinary prose extraction. I needed the system to preserve useful progress and explain failure, rather than make a book's value depend on an uninterrupted run.

There were also different kinds of work in the same journey. Uploading a large PDF moves bytes. Parsing prepares pages and their content. Structured extraction asks an external model to interpret a page. Summarisation produces a chapter-level result. Notification tells someone what happened. I separated those stages so that the API request did not need to stay open while the whole process finished.

The content had to serve applications beyond the upload screen. That meant the output shape mattered as much as the processing path. If each AI response came back in a different arrangement, the mobile app and school dashboard would inherit that uncertainty. I used constrained output and shared contracts to keep those applications working with an agreed format, rather than arbitrary text from a model.

I also needed a way to tell waiting from failure. A job retrying after a quota response is different from a job that has ended unsuccessfully. A book that is processing is different from an upload that has only reached storage. I made job states visible because the person using the system needs to understand those differences without interpreting worker logs.

I used signed upload links so the browser could send a PDF directly to object storage. The API coordinates the upload, but it does not carry the whole file through the application server. This keeps file transfer separate from the server's work of managing resources and jobs. It also gives the later workers a stored input to read, instead of tying processing to the original browser connection.

Completing the upload is the start of processing, not proof that the content is ready. I kept that distinction in the workflow. After storage receives the book, background work moves through parsing, processing, summarisation and notification. The interface can show the job state while those stages run. The user does not need to keep a long request open simply to let the book continue.

There is a trade-off in moving work behind a queue. The user no longer gets the final content in the upload response. The product has to show progress and failure separately. I accepted that because a long extraction process needs a lifecycle of its own. Pretending it is a quick form submission would only move the waiting and failure into a less visible place.

The parser reads the stored PDF and works through its pages. I added thumbnails and page records so the pipeline could deal with the book as more than an undivided file. Page numbers remain part of the content context. When a page needs attention, it is important to know where it belongs in the book, rather than have a collection of extracted text with no useful connection to the source.

Page-level progress was central to recovery. A book can contain work that has already completed alongside pages that still need processing. I built resumable handling so a stopped run does not have to mean discarding everything it achieved. The book remains the overall resource, while the page is a useful unit for recording progress and limiting the effect of a processing problem.

This is different from assuming that adding a queue makes the whole workflow recoverable. A queue can hold work, but the application still needs to know what happened inside that work. I put recovery around the stored page and book state as well as the queued jobs. That gives the next run something concrete to continue from when a worker has stopped.

I used vision-based extraction to read the page and return JSON-constrained output. The response has fixed block types rather than a free-form explanation of the page. That lets later stages distinguish prose, notation, tables and questions. I did not want each consumer to invent its own parser for whatever wording the model happened to return. The content service needed to carry that responsibility.

Academic notation needed particular care. An equation flattened into ordinary words can lose the relationship between symbols. The extraction instructions preserve mathematical notation in a renderable form. Tables and lists also have their own representation. These choices are about keeping the meaning available to later features, rather than making the extracted text merely look tidy in a preview.

I also kept extraction separate from summarisation. Reading a page should preserve its content. A chapter summary is a different output with a different purpose. If the extraction step rewrites the page into a short explanation, information is lost before the product has a chance to use it. I gave the later chapter work its own place in the pipeline instead of asking the page reader to do both jobs at once.

Images and diagrams are represented through reference blocks alongside their associated references. That gives the surrounding content a way to point to a figure without treating it as an ordinary paragraph. I avoided describing this as perfect understanding of every diagram. The useful part is that the content model has somewhere to represent that relationship, rather than silently removing it from the page structure.

I built retries with backoff for AI quota responses. Retrying immediately does not make an exhausted allowance available. It can keep the worker busy repeating the same failed request. Delayed retries give the external service time before another attempt. The job can show that it is retrying, so the interface does not have to describe every temporary problem as a completed failure.

I also added a hard time limit around page processing. A page that never returns should not occupy the worker indefinitely. A timeout gives the pipeline a defined way to stop that attempt and move into its failure or recovery handling. I kept the timeout as an operational bound, rather than a promise that every page would complete within a particular duration.

Retries and time limits solve different problems. A retry gives interrupted work another opportunity. A time limit prevents an attempt from waiting without end. Neither makes an external model infallible. I designed them alongside visible states and recovery paths because the product still needs a useful answer when repeated attempts do not produce a usable result.

The trade-off is more state to manage. A synchronous script can simply stop with an exception. A product pipeline has to describe whether it is waiting, processing, retrying or finished unsuccessfully. I accepted that extra work because it keeps a temporary processing problem from becoming an unexplained stalled book. The state is useful to both the worker and the person waiting for content.

A worker stopping mid-job leaves a different problem from an ordinary failed model call. The stored record can still say processing even though nothing is doing the work. I built startup recovery and handling for stuck books. The service can identify recoverable work and restart it, using the progress already recorded. That connects the state in storage with the actual ability to continue processing.

I did not treat every incomplete book as something that should be restarted immediately. Recovery needs a distinction between active work and stale work. The implementation uses stored activity when identifying books that may need recovery. This matters because restarting an active book would create another problem rather than repair the original one. Recovery belongs to the lifecycle, with its own conditions.

I also added fallback recovery around extraction. The important behaviour is that a difficult page does not require throwing away the whole book. Some problems still end in failure, and those need to remain visible. A system that keeps trying without showing the outcome is not enough. The operator needs to know which work completed and which part still needs another attempt or attention.

After page processing, a chapter worker produces summaries with constrained output. This gives the product a chapter-level result without replacing the extracted page content. I kept the summary as a derived output. The source material and the shorter explanation serve different needs, and keeping both lets the rest of the platform work with the form appropriate to the feature.

I added regeneration for failed chapters. If chapter processing needs another attempt, the operator can target that part rather than begin the whole upload journey again. This follows the same principle as saved page progress: make the recovery unit match the part that needs work. It is more useful than a broad restart control that does not explain what it will repeat.

Notifications and alerts report problems outside the worker logs. Logs remain useful for understanding a technical failure, but they are not a sufficient product interface for someone waiting on a book. I built a notification stage and operational alerts so failures could be noticed. That completes the workflow beyond successful extraction: the system also needs to communicate when content is not ready.

I built an API-key protected integration API used by the mobile app and school dashboard. Those applications can upload and process books through the same content system. The upload interface is therefore a client of a broader workflow, rather than the only way to use the pipeline. Other engineers can connect product features without moving the parser and worker logic into their applications.

I kept shared packages and queue contracts in the monorepo. This helped the API and workers agree about the work being passed between stages. The content model also gives consumers a defined set of block types. There is still integration work to do, but it can focus on using the result instead of reverse-engineering an unstructured model response.

I wrote tests around content normalisation as part of that boundary. The downstream product needs predictable data even when the incoming extraction has variations. For me, this was a useful distinction between generating an answer and building a content service. The service has to store, normalise, expose and recover its outputs, so another application can depend on more than a successful demonstration.