diff --git a/src/sandbox/dev_env.md b/src/sandbox/dev_env.md index c07dc15..3098834 100644 --- a/src/sandbox/dev_env.md +++ b/src/sandbox/dev_env.md @@ -97,9 +97,29 @@ Please carefully follow the terms and conditions at [Sandbox](https://sandbox.iu - all my data is gone after the semester break: all persistent storages get recycled of every(!) user each semester break. Please backup your data locally if needed - package conflicts: common issue is to install a unspecific library version, please specify or upgrade all dependencies manually. +## Large-Language-Models +An advanced Pytorch development environment is preinstalled with [Ollama](https://ollama.com/), which makes it easy to download and run different LLMs. + +If you want to run Ollama including the [OpenWebUI](https://openwebui.com/), following commands need to be executed in multiple Terminal windows: +![terminal](res/sandbox-terminal.png "Terminal") + +1. execute `ollama serve` to start the ollama backend +2. execute `ollama run mistral:7b` to download and run a specific model in a second terminal. + +Following steps are only needed if you want to access the WebUI. It also showcases how other http services can temporally be tunneled and exposed to public. +![tunnel](res/sandbox-tunnel.png "Tunnel") + +3. execute `open-webui serve` to serve the WebUI locally on port 8080 in another terminal. +4. visit [tun.iuk.hdm-stuttgart.de](https://tun.iuk.hdm-stuttgart.de) to obtain a token with your browser +5. execute `pgrok init --remote-addr tun.iuk.hdm-stuttgart.de:80 --forward-addr https://{user}.tun.iuk.hdm-stuttgart.de --token {token}` in the terminal. Replace `{user}` and `{token}` with your username and the previous obtained token. +6. execute `pgrok http 8080` to run the tunnel and expose the webui. Now you are able to access the webui at ´https://{user}.tun.iuk.hdm-stuttgart. + +### Notes +Please use the tunnel only temporally and carefully. The tunnel only support http(s) tunnels. Tunneled services are public available and accessible by anyone! If you want to train/finetune any LLMs, please use the [Training Environment](sandbox/training.md) instead. ## Useful Links - [Jupyter Documentation](https://docs.jupyter.org/en/latest/) - [pip](https://pip.pypa.io/en/stable/user_guide/) -- [python](https://docs.python.org/3.11/) \ No newline at end of file +- [python](https://docs.python.org/3.11/) +- [Ollama](https://ollama.com/) \ No newline at end of file diff --git a/src/sandbox/res/sandbox-terminal.png b/src/sandbox/res/sandbox-terminal.png new file mode 100644 index 0000000..36734a2 Binary files /dev/null and b/src/sandbox/res/sandbox-terminal.png differ diff --git a/src/sandbox/res/sandbox-tunnel.png b/src/sandbox/res/sandbox-tunnel.png new file mode 100644 index 0000000..0ab2708 Binary files /dev/null and b/src/sandbox/res/sandbox-tunnel.png differ