# Introducing the Pinecone Documents API > Pinecone's new Documents API models text, dense, and sparse fields in one schema. See what's new, what's changing, and how to migrate with the Python SDK. Jenna Pederson · 2026-09-09 Modeling anything beyond one vector per record has always meant bending your data to fit the index. Searchable text got flattened into a single field in a single sparse index, or dropped into metadata where it could be filtered but never ranked. Keyword search, if you needed it, usually meant standing up a second sparse index. Then reconciling two sets of results yourself, by hand. The new Documents API changes that. It's an index defined by a schema that holds text fields, dense vectors, sparse vectors, and metadata together on the same document, and it's queried through a new set of APIs built around documents rather than vectors. The Documents API is what makes [full-text search](https://www.pinecone.io/blog/full-text-search-generally-available/) possible. Both are generally available today. This post covers what the Documents API is, why we built it, what it means for an existing index, and what a schema-based index looks like in code. The examples below use engineering incident postmortems as a running example: a narrative field, a short summary field, a dense vector for semantic search over the narrative, and metadata like severity and team. ## What's new This ships in [Python SDK 10.0.0](https://docs.pinecone.io/reference/sdks/python/overview). Creating an index now takes a schema. A schema declares the fields on an index up front: one or more text fields (each independently configurable for full-text search), a dense vector field, and a sparse vector field. Anything upserted that isn't declared in the schema is stored and automatically indexed as filterable metadata, the same as before. ``` from pinecone import Pinecone, SchemaBuilder pc = Pinecone(api_key="YOUR_API_KEY") schema = ( SchemaBuilder() .add_string_field("narrative", full_text_search={"language": "en"}) .add_string_field("summary", full_text_search={"language": "en"}) .add_dense_vector_field("embedding", dimension=1536, metric="cosine") .build() ) pc.indexes.create( name="incident-postmortems", schema=schema, ) ``` Working with a schema-based index uses a new set of methods, scoped to documents instead of vectors: ``` index = pc.Index("incident-postmortems") index.documents.upsert(documents=[...]) index.documents.search(score_by=[...]) ``` Everything here is written against the Python SDK. The same capabilities are available in the [TypeScript SDK](https://docs.pinecone.io/reference/sdks/node/overview). ## Why we built it The Documents API is a different way of modeling data on Pinecone, and the foundation for capabilities beyond what ships today. Start from what an agent actually needs. An agent trying to complete a task rarely wants "the ten nearest vectors" on their own. It wants specific documents: matched by exact terms, filtered by metadata, ranked by relevance, often all at once, with text carrying as much weight in that mix as vectors do. One schema, every field type. A schema set at index creation can declare multiple independent text fields alongside a dense vector field, a sparse vector field, and any amount of filterable metadata, all on the same document. `narrative` and `summary` can each be their own field, scored or filtered on its own terms, instead of getting flattened into one blob because that's all the index had room for. Keyword search and vector search run against that same schema, in the same index, so there’s nothing extra to stand up or reconcile. That's one index to provision, monitor, and pay for instead of two, and one call to get a result back instead of a client-side merge step in between. ## What stays the same An existing vectors-API index keeps working. Upgrading the SDK doesn't move it to a new data plane, and its `query`, `upsert`, `fetch`, `update`, `delete`, and `list` methods keep working unchanged against it. Integrated embedding and the records API are unaffected entirely. Creating a _new_ vectors-API index also works the same way it always has: ``` pc.create_index( name="incident-postmortems", vector_type="dense", dimension=1536, metric="cosine", spec=ServerlessSpec(cloud="aws", region="us-east-1"), ) ``` It's queried and updated with the same `query`, `upsert`, `fetch`, `update`, `delete`, and `list` methods as any other legacy index. ## Breaking changes to know about ### Dense and sparse fields must be declared explicitly If you're building a schema with both a dense and a sparse field, declare both explicitly. On the old vectors API, `metric="dotproduct"` alone implied a sparse-capable index. On a schema-based index, a `sparse_vector` field has to be added to the schema at creation time, or upserting sparse values later will fail. ### Reading index metadata Index-management calls like `describe_index` and `list_indexes` return updated shapes after an SDK upgrade, for indexes of either kind. Fields like `.dimension` and `.metric` move under .`schema.fields[...]`: ``` # Before index_info.dimension # After index_info.schema.fields["embedding"].dimension ``` `dimension`, for example, only appears when an index has a dense vector field. Refer to the [guide for adopting the Documents API](https://docs.pinecone.io/guides/index-data/adopt-the-documents-api) for more details. ## Before and after ### Creating the index With the vectors API, creating an index meant specifying a `vector_type`, `dimension`, `metric`, and `spec`. Now, a schema-based index replaces all four with a single `schema`: ``` schema = ( SchemaBuilder() .add_string_field("narrative", full_text_search={"language": "en"}) .add_string_field("summary", full_text_search={"language": "en"}) .add_dense_vector_field("embedding", dimension=1536, metric="cosine") .build() ) pc.indexes.create( name="incident-postmortems", schema=schema, deployment={"deployment_type": "managed", "cloud": "aws", "region": "us-east-1"}, ) ``` The schema declares `narrative` and `summary` as two independent text fields alongside the dense vector field named `embedding`. ### Upserting With the vectors API, an upsert meant a vector plus a `metadata` dict, with anything filterable or human-readable (like the narrative text) stuffed into that dict. On a schema-based index, an upsert is a document: the schema fields and any extra fields sit side by side, with no separate `metadata` parameter to reason about. ``` index.documents.upsert( documents=[ { "_id": "inc-4471", "narrative": "Checkout latency spiked after a config rollout...", "summary": "Checkout latency regression", "embedding": embedding, "severity": "sev2", "team": "payments", } ], ) ``` Fields declared in the schema (`narrative`, `summary`, `embedding`) are validated against it; anything else (`severity`, `team`) is stored and indexed for filtering automatically. ### Querying On the old vectors API, `narrative` could live in metadata, filterable but with no ranking, or get encoded into a sparse vector for lexical scoring, real ranking but no native BM25. On a schema-based index, it's a real text field with BM25 ranking behind it: ``` index.documents.search( top_k=10, score_by=[{"type": "text", "field": "narrative", "query": "checkout latency"}], filter={"severity": {"$match_phrase":"sev2"}}, include_fields=["narrative", "severity"], ) ``` This shows a single scoring method on one field alongside metadata filtering that is applied before scoring. This is meant to show the shape of the new query, not the full range of what's possible. The [full-text search guide](https://docs.pinecone.io/guides/search/full-text-search) covers the rest: scoring by dense or sparse vectors, Lucene query syntax, and the filter operators available on text fields. ## Migrating an existing index Moving an index onto a schema is opt-in and doesn't happen on its own. Nothing about an existing index changes until this is done deliberately: 1. Upgrade the python SDK to 10.0.0 or later. 2. Create a new index with the schema you want. 3. Reindex your data into it. The [guide for adopting the Documents API](https://docs.pinecone.io/guides/index-data/adopt-the-documents-api) steps through each step in detail. ## FAQ **Do I need to change anything about my existing index?** No. It keeps working as-is. **Does this affect integrated embedding or the records API?** No. Integrated embedding on schema-based (document) indexes is coming. **Can I still create vectors-API indexes going forward?** Yes, just make sure to use the vectors-API `query`, `upsert`, `fetch`, `update`, `delete`, and `list` methods. **Can I convert an existing index to a schema-based one?** No. There's no in-place conversion. A new schema means a new index and a reindex. More questions, including hybrid search, integrated inference, and pinning API versions, are answered in the [guide for adopting the Documents API](https://docs.pinecone.io/guides/index-data/adopt-the-documents-api). ## Wrapping up For a team building keyword and vector search together, the Documents API means no more running a separate sparse index to keep in sync with the dense one, and no more merging two sets of results by hand. It's one schema, one index, and it's what makes full-text search, generally available today, possible in the first place. For more, see the [guide for adopting the Documents API](https://docs.pinecone.io/guides/index-data/adopt-the-documents-api), the [full-text search guide](https://docs.pinecone.io/guides/search/full-text-search), and the [data modeling guide](https://docs.pinecone.io/guides/index-data/data-modeling).