Installation
Install from a C++ Source
To build and install the SentencePiece command line tools from C++ source on Ubuntu, use the following code:
% sudo apt-get install cmake build-essential pkg-config libgoogle-perftools-dev
% git clone https://github.com/google/sentencepiece.git
% cd sentencepiece
% mkdir build
% cd build
% cmake ..
% make -j $(nproc)
% sudo make install
% sudo ldconfig -v
Install from a Python Module
pip install sentencepiece
Please ensure that the SentencePiece Python module is pre-installed in the NeMo Framework container.
For more information on installation, please see this page.