core.inference.text_generation_server.run_mcore_engine#
Module Contents#
Functions#
Server-compatible version of the MCore Engine, used in tools/run_text_generation_server.py. |
API#
- core.inference.text_generation_server.run_mcore_engine.run_mcore_engine(
- engine,
- prompts=None,
- temperature=1.0,
- top_k=0,
- top_p=0.0,
- logprobs=True,
- tokens_to_generate=0,
- top_n_logprobs=0,
- random_seed=-1,
Server-compatible version of the MCore Engine, used in tools/run_text_generation_server.py.