The gap between a working Jupyter notebook and a deployed production system is wider than most ML practitioners appreciate when they first encounter it.
A notebook is an exploration environment. It holds state implicitly variables persist, models sit in memory, the tokenizer that was fit on training data is available because you ran that cell twenty minutes ago. None of this survives restart. None of it is callable from another system. None of it can be versioned, tested, or monitored.
The Fake News Detection system started as a notebook a BiLSTM classifier trained on the LIAR dataset, achieving strong accuracy on a binary fake/real classification task. The project's real engineering challenge was not the model. It was the transition: turning that notebook into a production inference service that any application can call.
Why BiLSTM for This Problem
The choice of Bidirectional LSTM over simpler baselines TF-IDF + Logistic Regression, for instance reflects a specific hypothesis about what makes news text misleading.
Fake news rarely fails at the word level. The vocabulary is largely the same as real news. What differs is structure: how sentences are constructed, what rhetorical patterns appear, how claims are sequenced and qualified. These are sequential dependencies that bag-of-words models cannot capture.
A standard LSTM reads text left to right, building a hidden state that encodes the history of the sequence. For news classification, right-to-left context matters too the resolution of a claim, the hedging language that qualifies an earlier assertion, the conclusion that reframes the opening. A BiLSTM processes the sequence in both directions and concatenates the resulting representations, giving the classifier access to both forward and backward sequential context.
The practical result is a model that reasons about the full structure of a text, not just its component vocabulary.
The Production Architecture Problem
When a trained model lives in a notebook, three implicit dependencies exist that become explicit engineering problems in production:
The tokenizer. During training, text is tokenized converted to integer sequences based on a vocabulary learned from the training corpus. That vocabulary must be identical at inference time. If the tokenizer is refit from scratch, or if a different tokenizer is used, the model receives different integer sequences than it was trained on and produces meaningless output. The solution is to serialize the fitted tokenizer as an artifact in this case using Joblib and load it alongside the model at startup.
The sequence length. The BiLSTM was trained on padded sequences of a fixed maximum length. Input at inference time must be padded or truncated to the same length, with the same padding strategy (pre-sequence or post-sequence). This is not a hyperparameter the serving code can guess. It must be codified.
The model format. TensorFlow models can be saved in multiple formats. The .keras format used here preserves the full model architecture, weights, and compilation configuration in a single artifact. This is the correct choice for a production deployment it requires no reconstruction of the model graph and loads deterministically across environments.
These three artifacts model, tokenizer, sequence length define the inference contract. The FastAPI service loads all three at startup via a lifespan context manager, making them available to every prediction request without reloading.
The API Layer
The API exposes two endpoints. The health endpoint confirms that the service is running and that the model loaded successfully. The prediction endpoint accepts a text string and returns a classification label (Fake or Real) and a confidence score between 0 and 1.
{
"text": "Breaking news content goes here..."
}
{
"confidence": 0.87,
"label": "Fake"
}
The simplicity of this interface is deliberate. The consumer of this API whether a React frontend, a browser extension, a content moderation pipeline, or a news aggregation service should not need to know anything about BiLSTMs, tokenization, or sequence padding. Those concerns belong entirely inside the service boundary.
Pydantic schemas enforce the input contract. The preprocessing pipeline, text normalization, tokenization, padding runs inside the predictor module, invisible to the API consumer. The model's raw output (a sigmoid probability) is mapped to a human-readable label with a calibrated confidence score.
The Preprocessing Layer
The preprocessing module deserves attention because it is where most production NLP systems fail silently.
The tokenizer learned during training was fit on specific text with specific cleaning applied before tokenization. If the preprocessing applied at inference time differs from preprocessing applied at training time, the tokenizer receives different input than it was designed for. Words that were split or normalized during training will be seen as unknown tokens at inference.
The preprocessing pipeline here applies consistent text normalization: lowercasing, punctuation handling, whitespace normalization. This is applied identically during training artifact creation and at inference. The serialized tokenizer encodes the vocabulary that results from that normalized text, ensuring that the integer sequences produced at inference match the distribution the model was trained on.
The Containerization Decision
The Dockerfile uses a multi-stage build approach. The build stage installs dependencies. The production stage copies only what is needed for inference the model artifact, the tokenizer, the application code and runs with a non-root user.
This matters for three reasons. Image size: a full TensorFlow installation without careful exclusion of development dependencies produces images exceeding 2GB. Build reproducibility: pinned dependency versions in requirements.txt ensure the same environment regardless of when the image is built. Security posture: running inference as a non-root user with minimal permissions reduces the attack surface of the deployed service.
The containerized service can deploy to any cloud provider that supports Docker Render, Railway, AWS ECS, Google Cloud Run, Azure Container Instances without environment-specific configuration changes. The service discovers its configuration through environment variables, and the model artifacts are baked into the image rather than fetched at runtime, eliminating the startup latency and dependency on external storage.
What This System Gets Right About ML Production
Three principles govern the architecture of this system that apply broadly to production ML:
Artifacts are the contract. The model file and the tokenizer file, versioned together, define what the system knows and how it processes input. Changing either requires a new deployment. This is not a limitation it is the correct way to think about model versioning. An API version should correspond to a specific pair of model and preprocessing artifacts.
The serving layer does not do research. The preprocessing, tokenization, and inference pipeline in predictor.py contains no exploratory code, no print statements, no conditional logic for different experiments. It does exactly one thing: given a text string, return a label and confidence. Mixing research code into serving code is how production systems become fragile and unmaintainable.
The API boundary is a trust boundary. Everything inside the API model, tokenizer, preprocessing, confidence calculation is the system's responsibility. Everything outside what text is submitted, how the response is displayed, what threshold the consumer uses to act on the confidence score is the consumer's responsibility. The boundary is clean, explicit, and documented in the schema.
These are not complex principles. They are the principles that distinguish deployed ML systems from notebooks that happen to have a web framework in front of them.
Full source code and API documentation are available on GitHub.