Skip to content

Commit

Permalink
fix: Resolve Milvus connection issue, add config to milvus test, add …
Browse files Browse the repository at this point in the history
…milvus gh action

Resolved if statement resolution issue regrading api key,
Added vector db config to milvus test,
Added milvus gh action

Fix
  • Loading branch information
dexters1 committed Dec 3, 2024
1 parent fb5f0cf commit 764c089
Show file tree
Hide file tree
Showing 4 changed files with 85 additions and 12 deletions.
64 changes: 64 additions & 0 deletions .github/workflows/test_milvus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: test | milvus

on:
workflow_dispatch:
pull_request:
branches:
- main
types: [labeled, synchronize]

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
RUNTIME__LOG_LEVEL: ERROR
ENV: 'dev'

jobs:
get_docs_changes:
name: docs changes
uses: ./.github/workflows/get_docs_changes.yml

run_milvus:
name: test
needs: get_docs_changes
if: needs.get_docs_changes.outputs.changes_outside_docs == 'true' && ${{ github.event.label.name == 'run-checks' }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
defaults:
run:
shell: bash

steps:
- name: Check out
uses: actions/checkout@master

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.11.x'

- name: Install Poetry
# https://github.com/snok/install-poetry#running-on-windows
uses: snok/[email protected]
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true

- name: Install dependencies
run: poetry install -E milvus --no-interaction

- name: Run default basic pipeline
env:
ENV: 'dev'
LLM_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: poetry run python ./cognee/tests/test_milvus.py

- name: Clean up disk space
run: |
sudo rm -rf ~/.cache
sudo rm -rf /tmp/*
df -h
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
from __future__ import annotations

import asyncio
import logging
from typing import List, Optional
Expand All @@ -6,7 +8,6 @@
from ..vector_db_interface import VectorDBInterface
from ..models.ScoredResult import ScoredResult
from ..embeddings.EmbeddingEngine import EmbeddingEngine
from pymilvus import MilvusClient

logger = logging.getLogger("MilvusAdapter")

Expand All @@ -31,8 +32,9 @@ def __init__(self, url: str, api_key: Optional[str], embedding_engine: Embedding

self.embedding_engine = embedding_engine

def get_milvus_client(self) -> MilvusClient:
if self.api_key is not None:
def get_milvus_client(self) -> "MilvusClient":
from pymilvus import MilvusClient
if self.api_key:
client = MilvusClient(uri=self.url, token=self.api_key)
else:
client = MilvusClient(uri=self.url)
Expand Down
8 changes: 8 additions & 0 deletions cognee/tests/test_milvus.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,14 @@ async def main():
pathlib.Path(os.path.join(pathlib.Path(__file__).parent, ".cognee_system/test_milvus")).resolve())
cognee.config.system_root_directory(cognee_directory_path)

cognee.config.set_vector_db_config(
{
"vector_db_url": os.path.join(cognee_directory_path, "databases/milvus.db"),
"vector_db_key": "",
"vector_db_provider": "milvus"
}
)

await cognee.prune.prune_data()
await cognee.prune.prune_system(metadata=True)

Expand Down
17 changes: 8 additions & 9 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 764c089

Please sign in to comment.