Manage data

In addition to inserting and querying data, there are other ways you can interact with vector data in a Pinecone index. This section walks through the various vector operations available.

Connect to an index

If you're using a Pinecone client library to access an index, you'll need to open a session with the index:

pythoncurl
Copy
Copied
# Connect to the index
index = pinecone.Index("pinecone-index")
Copy
Copied
# Not applicable

Pinecone indexes each have their own DNS endpoint.

Describe index statistics

Get statistics about an index, such as vector count per namespace:

pythoncurl
Copy
Copied
index.describe_index_stats()
Copy
Copied
curl -i -X GET https://YOUR_INDEX-PROJECT_NAME.svc.YOUR_ENVIRONMENT.pinecone.io/describe_index_stats \
  -H 'Api-Key: YOUR_API_KEY'

Fetching vectors

The Fetch operation looks up and returns vectors, by id, from an index. The returned vectors include the vector data and/or metadata. Typical fetch latency is under 5ms.

Fetch items by their ids:

pythoncurl
Copy
Copied
index.fetch(["id-1", "id-2"])

# Returns:
# {'namespace': '',
#  'vectors': {'id-1': {'id': 'id-1',
#                       'values': [0.568879, 0.632687092, 0.856837332, ...]},
#              'id-2': {'id': 'id-2',
#                       'values': [0.00891787093, 0.581895, 0.315718859, ...]}}}
Copy
Copied
curl -i -X GET "https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/fetch?ids=id-1&ids=id-2" \
  -H 'Api-Key: YOUR_API_KEY'
# Output:
# {
#   "vectors": {
#     "id-1": {
#       "id": "id-1",
#       "values": [0.568879, 0.632687092, 0.856837332, ...]
#     },
#     "id-2": {
#       "id": "id-2",
#       "values": [0.00891787093, 0.581895, 0.315718859, ...]
#     }
#   },
#   "namespace": ""
# }

Updating vectors

There are two methods for updating vectors and metadata, using full or partial updates.

Full update

Full updates modify the entire item, that is vectors and metadata. Updating an item by id is done the same way as inserting items. (Write operations in Pinecone are idempotent.)

The Upsert operation writes vectors into an index.

note

If a new value is upserted for an existing vector id, it will overwrite the previous value.

  1. Update the value of the item ("id-3", [3.3, 3.3]):
pythoncurl
Copy
Copied
index.upsert([("id-3", [3.3, 3.3])])
Copy
Copied
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/upsert \
  -H 'Api-Key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "vectors": [
      {
        "id": "id-0",
        "values": [3.3, 3.3]
      }
    ]
  }'
  1. Fetch the item again. We should get ("id-3", [3.3, 3.3]):
pythoncurl
Copy
Copied
index.fetch(["id-3"])
Copy
Copied
curl -i -X GET https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/fetch?ids=id-3 \
  -H 'Api-Key: YOUR_API_KEY'

Partial update

The Update operation performs partial updates that allow changes to part of an item. Given an id, we can update the vector value with the values argument or update metadata with the set_metadata argument.

warning

The Update operation does not validate the existence of ids within an index. If a non-existent id is given then no changes are made and a 200 OK will be returned.

To update the value of item ("id-3", [3., 3.], {"type": "doc", "genre": "drama"}):

pythoncurl
Copy
Copied
index.update(id="id-3", values=[4., 2.])
Copy
Copied
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/update \
  -H 'Api-Key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "vectors": [
      {
        "id": "id-3",
        "values": [4., 2.]
      }
    ]
  }'

The updated item would now be ("id-3", [4., 2.], {"type": "doc", "genre": "drama"}).

When updating metadata only specified fields will be modified. If a specified field does not exist, it is added.

note

Metadata updates apply only to fields passed to the set_metadata argument. Any other fields will remain unchanged.

To update the metadata of item ("id-3", [4., 2.], {"type": "doc", "genre": "drama"}):

pythoncurl
Copy
Copied
index.update(id="id-3", set_metadata={"type": "web", "new": "true"})
Copy
Copied
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/update \
  -H 'Api-Key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "vectors": [
      {
        "id": "id-3",
        "set_metadata": {"type": "web", "new": "true"}
      }
    ]
  }'

The updated item would now be ("id-3", [4., 2.], {"type": "web", "genre": "drama", "new": "true"}).

Both vector and metadata can be updated at once by including both values and set_metadata arguments. To update the "id-3" item we write:

pythoncurl
Copy
Copied
index.update(id="id-3", values=[1., 2.], set_metadata={"type": "webdoc"})
Copy
Copied
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/update \
  -H 'Api-Key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "vectors": [
      {
        "id": "id-3",
        "values": [1., 2.],
        "set_metadata": {"type": "webdoc"}
      }
    ]
  }'

The updated item would now be ("id-3", [1., 2.], {"type": "webdoc", "genre": "drama", "new": "true"}).

Deleting vectors

The Delete operation deletes vectors, by id, from an index.

Alternatively, it can also delete all vectors from an index or namespace.

note

If you delete all vectors from a single namespace, it will also delete the namespace.

To delete vectors by their ids:

pythoncurl
Copy
Copied
index.delete(ids=["id-1", "id-2"], namespace='example-namespace')
Copy
Copied
curl -i -X DELETE "https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io.pinecone.io/vectors/delete?ids=id-1&ids=id-2&namespace=example-namespace" \
  -H 'Api-Key: YOUR_API_KEY'

To delete all vectors from a namespace:

pythoncurl
Copy
Copied
index.delete(delete_all=True, namespace='example-namespace')
Copy
Copied
curl -i -X DELETE "https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/delete?delete_all=true&namespace=example-namespace" \
  -H 'Api-Key: YOUR_API_KEY'