Limitations of mini-assistant

[RobinQu]

This is long live issue that tracks limitations of mini-assistant implementation.

General speaking, mini-assistant is an all-in-one, single-node jukebox that mimick OpenAI's Assistant API. It's not intended for large-scale and distributed production system.

When mini-assistant is matured enought, and the community actually demands a more power version, I will start to work on mighty-assitant submodule.

Related issues:

• First version of file-search tool for assistant-api #20
• Modular RAG implementations: reranker, multi-retrievers #16

[RobinQu]

General and tool-use

• Not yet implemented
  • Stream API
  • Context compression
  • Properties like tool_resources, temperature are working only with Assistant. Some may not work on Thread or Run. A more detailed description will be provided.
• In-consistency with OpenAI
  • Agent trajectories may differ, as we take completely different implementations. And OpenIA's technical details are beyond my reach.

file-search

• Not yet implemented
  • File expiration policy
  • Only pdf, markdown, and txt are supported right now. But further support is relatively easy.
  • Upload limitations in both file count and file bytes.
  • DB transaction is required for some APIs, but not currently implemented. This may cause some in-consistency in some corner cases, which will be addressed in near future.
• In-consistency with OpenAI
  • In file annotations, as start_index and end_index are implementation dependent, it cannot be exact the same if we don't have access to the OpenAI's code.
  • The way file search is activated is not documented by OpenAI. It's largely dependent on how agent workflow is implemented. In mini-assistant, we first decide which files are most relevant to the user question, and do recall within these files. So it's expected that some queries won't trigger file search at all.
  • Due to the current implementation, extra time is needed to generate summary for each uploaded file.

code interpreter

This is not yet implemented.