`core.inference.text_generation_server.dynamic_text_gen_server.endpoints.common`#

Module Contents#

Functions#

send_do_generate

Broadcasts a message to perform a generation to all tensor parallel ranks.

Data#

`GENERATE_NUM`
`LOCK`

API#

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.common.GENERATE_NUM#: 0

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.common.LOCK#: ‘Lock(…)’

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.common.send_do_generate()#: Broadcasts a message to perform a generation to all tensor parallel ranks.