# NLP

> Natural language processing uses computational methods to analyse, transform, and generate human language.

- HTML version: https://robbiepalmer.me/ideas/nlp
- Source: https://en.wikipedia.org/wiki/Natural_language_processing

Natural language processing applies computational methods to human language. Tasks include
classification, named-entity recognition, information extraction, search, translation, and text
generation. Modern NLP includes rules, statistical models, specialist language models, and general
large language models.

Language is ambiguous and depends on context. The same phrase can name a person, product, place, or
identifier. A useful system therefore needs a defined task, examples from its real domain, and an
evaluation tied to the downstream decision rather than a generic language benchmark.

In [Intelligent Document Processing](/ideas/intelligent-document-processing), NLP can map recognised
text to fields and entities after [OCR](/ideas/ocr) has recovered the characters. It can also work
directly on born-digital text, as in entity detection over messages or intent recognition in a
conversational interface.

## Questions it prompts

* What language decision does the product need to make?
* Which vocabulary, ambiguity, and document types occur in the real domain?
* What should happen when the model lacks enough context?
* Which evaluation errors would change a downstream action?

## Related ideas

- [Intelligent Document Processing](https://robbiepalmer.me/ideas/intelligent-document-processing.md): A pipeline that turns varied documents into validated, structured data by combining recognition, language understanding, and workflow rules.
- [Named-Entity Recognition](https://robbiepalmer.me/ideas/ner.md): NER finds spans of text that name entities and assigns them types such as people, organisations, products, or identifiers.

## Where it appears

- Project: [AI Shopping Assistant](https://robbiepalmer.me/projects/chatbot.md)
- Project: [Commercial Knowledge Graph](https://robbiepalmer.me/projects/commercial-knowledge-graph.md)
- Project: [Intelligent Document Processing](https://robbiepalmer.me/projects/intelligent-document-processing.md)
- Project: [Automated PII Detection](https://robbiepalmer.me/projects/pii-detection.md)
- Project: [Recipe Site](https://robbiepalmer.me/projects/recipe-site.md)
- Blog post: [How To Automatically Detect PII for Real-Time Cyber Defense](https://robbiepalmer.me/blog/2023-05-23-automatically-detect-pii-real-time-cyber-defense.md)
- ADR: [ADR 031: OpenRouter](https://robbiepalmer.me/projects/recipe-site/adrs/031-openrouter.md)

---

Markdown index of this site: https://robbiepalmer.me/llms.txt