`core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server`#

Module Contents#

`temp_log_level`	Enables temporarily overriding the logging level.
`_run_text_gen_server`	Initializes and runs the async web server. Automatically starts and manages its own InferenceClient connected to the provided coordinator address.
`_server_process_worker`	Synchronous worker function that sets up a new event loop for the separate process.
`start_text_gen_server`	Start the text generation server.
`stop_text_gen_server`	Stop the text generation server.

core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server.logger#: ‘getLogger(…)’

core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server._SERVER_PROCESSES: List[multiprocessing.Process]#: []

core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server._SHARED_SOCKET#: None

core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server.temp_log_level(level, logger=None)#: Enables temporarily overriding the logging level.

async core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server._run_text_gen_server( coordinator_addr: str, tokenizer, rank: int, server_port: int, parsers: Optional[List[str]] = None, verbose: bool = False, fd: Optional[int] = None, hostname: Optional[str] = None, )#: Initializes and runs the async web server. Automatically starts and manages its own InferenceClient connected to the provided coordinator address.

core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server._server_process_worker( coordinator_addr: str, tokenizer, rank: int, server_port: int, parsers: Optional[List[str]] = None, verbose: bool = False, fd: Optional[int] = None, hostname: Optional[str] = None, )#: Synchronous worker function that sets up a new event loop for the separate process.

core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server.start_text_gen_server( coordinator_addr: str, tokenizer, rank: int, server_port: int, parsers: Optional[List[str]] = None, verbose: bool = False, num_replicas: int = 4, hostname: Optional[str] = None, )#: Start the text generation server.

core.inference.text_generation_server.dynamic_text_gen_server.text_generation_server.stop_text_gen_server()#: Stop the text generation server.