FixedSizeList is not the same. input list: fixed_size_list<item: float>[3072] output list: fixed_size_list<item: float>[4096]

[AhmadShaik]

Exception has occurred: ArrowTypeError
Size of FixedSizeList is not the same. input list: fixed_size_list<item: float>[3072] output list: fixed_size_list<item: float>[4096]
File "/media/ahamad/work/projects/auto_zone/Autozone_experiments/knowledge_base.py", line 47, in
knowledge_base.load(recreate=False)
pyarrow.lib.ArrowTypeError: Size of FixedSizeList is not the same. input list: fixed_size_list<item: float>[3072] output list: fixed_size_list<item: float>[4096]

[ysolanky]

Hello @AhmadShaik! Can you please share your Agent config including the config for the knowledge base?

[AhmadShaik]

@ysolanky - here is the agent config and the knowledge base config. The error poppedup at the knowledge_base.load()

txt_path = "/directory/path/to/files"
# # html_paths = get_sds_paths(pdf_dir, extn=".html")  
# chunking_strategy=SemanticChunking()

vector_db = LanceDb(
    table_name="auto_zone",
    uri="/tmp/lancedb",
    embedder=OllamaEmbedder()
)

knowledge_base = PDFKnowledgeBase(
    path=txt_path,
    vector_db=vector_db,     
    # vector_db=ChromaDb(collection="auto_zone", embedder=OllamaEmbedder()),
    # chunking_strategy=SemanticChunking()
    chunking_strategy=FixedSizeChunking()
)

knowledge_base.load(recreate=False)

agent = Agent(
    knowledge_base=knowledge_base,
    search_knowledge=True,
)