DefendaMe: documents, AI and operations

In the local PDF workflow, page images had to remain on the user’s device because they could contain sensitive material. Extracted text could be sent to the cloud for search and analysis. That distinction shaped the design: processing images locally did not mean keeping every part of document processing out of the cloud.

We initially chose to run OCR in the browser. Our user was a nontechnical lawyer; we wanted to avoid requiring another application to use a web system. We used PDF.js to prepare pages and Tesseract.js to recognize text.

The cost of this choice appeared during execution. In our experience, processing used substantial memory and sometimes caused the browser to close. Clients with less capable machines struggled even more. Avoiding installation did not remove dependence on the device.

The team adjusted concurrency to the machine’s capacity, processed pages in batches, released rendering resources and recreated workers between batches. These workers ran in the browser; they were different from backend queue workers. We also implemented incremental saving of results in IndexedDB, the browser’s local storage database.

Saving intermediate results preserved some work, but did not guarantee automatic recovery from every interruption. Recovery also depended on access to the original content and reprocessing paths that were still incomplete.

After extraction, there was another boundary: indexing the text. We implemented per-page tracking, retries after failures and stable identifiers in Elasticsearch. Indexing the same page again should not create a second document. This protection applied to indexing; it did not make every effect in the pipeline idempotent. My contribution to this work included reprocessing failed pages.

The browser removed the initial installation step, but retained memory and execution constraints we had to manage. I consider a desktop application a possible next direction for more control over local processing. It remains an alternative to validate, including installation and maintenance, rather than a delivery described here.

Legal reports needed to follow a schema validated with Zod, often with lengthy JSON responses. There were three separate requirements: the JSON had to be parseable, its fields had to follow the schema, and its content needed legal quality. Passing structural validation did not demonstrate that the analysis was correct.

We evaluated models against these requirements using the product’s workloads. When structure failed, repairing or repeating generation added time and cost. Price per call was therefore insufficient: what mattered was the cost of obtaining a usable report, including discarded attempts and corrections.

During this process, we moved from Gemini 2.5 Pro to GPT-5. The switch did not eliminate failures. I worked on recovery of invalid responses through sanitization, correction and further validation. The implementation later evolved to repair only the problematic fields instead of indiscriminately regenerating the entire response.

There were also compatibility adjustments between schemas and the Structured Outputs API. Part of the difficulty was in the integration, rather than the model alone. These changes support an account of a model choice accompanied by recovery work, rather than a conclusion about the entire Gemini family or a guarantee of legally correct output from the adopted models.

The trade-off was retaining validation and recovery in the system even after changing the model. We do not have a reproducible comparison that quantifies how much this choice reduced failures or total cost. The decision was to evaluate responses we could actually use, rather than the advertised price per token alone.

I directed the adoption of Sentry to better understand failures during product use, with Ryan participating in the implementation. We combined browser error capture with events from stages such as upload, OCR and analysis, at instrumented points in the client and server. Context included the operation in progress and start, completion or failure events.

I later worked on the evolution of structured JSON logs and correlation between executions. Depending on the instrumented point, records included execution, service, stage, duration and attempt. A correlation identifier connected events that would otherwise require reading isolated messages.

This added mechanisms for investigating processing, but did not provide complete tracing of every path or demonstrate a measured reduction in diagnosis time. The result I can describe here is the implemented instrumentation and its evolution.

Useful investigation context also needed limits. We applied masking and filters to restrict the transmission of sensitive content. More telemetry required more care with collection; these mechanisms were not a guarantee of complete anonymization.

We used Elasticsearch for text search and PostgreSQL for vector search. The application also ran the embedding model over the texts, sharing processing and memory with product features.

The application fed two search layers and ran the embedding model.

Since Elasticsearch was already part of the infrastructure, we moved responsibility for embedding generation to the ingestion flow associated with search. The application began sending text, and we brought text and vector search together in Elasticsearch.

The computational cost of inference still existed. The change removed model execution from the application server and concentrated indexing in the search infrastructure. It also increased our dependence on that infrastructure.

The diagrams represent this transfer of responsibility. They do not detail the inference mechanism or internal topology.

Running embeddings, indexing and analysis could take many minutes. Accommodating these workloads in Vercel’s execution model was becoming inconvenient. The team decided to move to a DigitalOcean Droplet; the web layer moved as well because, in our assessment, the machine’s fixed cost was more convenient than the platform’s per-seat cost.

I was among those who most strongly argued for staying on Vercel because of the managed operations work it handled. We reached an agreement to migrate, with my reservations still in place. The decision and the work of dealing with its consequences were collective.

We gained control over processes and dependencies, but became directly responsible for deployment, the proxy, communication between containers, memory and process recovery. We encountered problems in these layers and spent time investigating them. For a small team, this work competed for attention with product features.

In retrospect, I consider moving the entire application a mistake. I would have preferred keeping the web layer on Vercel and moving only the services that needed a different execution model. This is my assessment today, rather than an alternative whose cost or availability we measured under equivalent conditions.

Configuration brought another concrete problem. The application had accumulated hundreds of environment variables, including configuration and secrets, and differences between environments were among the bugs we frequently encountered. We evaluated alternatives, particularly HashiCorp Vault, and chose Doppler for its relative simplicity in our context. I contributed directly to deployment integration, using the CLI to provide variables to the process.

Centralizing configuration did not guarantee that it reached each service. A later fix addressed a variable available in Doppler that Compose was not passing into a container. The integration changed configuration management; we remained responsible for checking its propagation.