grosse
/
sandbox-docs-public
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

added docs
ci/woodpecker/push/woodpecker Pipeline was successful Details

This commit is contained in:
maltegrosse 2023-07-14 15:35:40 +02:00
parent f02921d667
commit 2eedd31b2a
4 changed files with 46 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/sandbox/data_generator.md
View File

@ -1,6 +1,4 @@
# Data Generator
Coming Soon ...
Currently under heavy development, coming soon....stayed tuned!

22
src/sandbox/data_pool.md
View File

@ -1,17 +1,14 @@
# Data Pool
# Storage
In the following section we describe how to store data on the Sandbox. There are three diffrent ways to do so: Inside the Sandbox, Object Storage, Git LFS.
In the following section we describe how to store data on the Sandbox. There are three different ways to do achieve: Inside the Sandbox, headless file upload on Object storage and Git LFS.
## Inside the Sandbox
To store the data inside the Sandbox, you just have to drag & drop or click on the upload button to save the file to your running instance. You can also create folders and new Notebooks. The only limitation is, that each user has 1GB of storage.
To store the data inside the Sandbox, you just have to drag & drop or click on the upload button to save the file to your running instance. You can also create folders and new Notebooks. Please see the free space limitations [here](sandbox/dev_env#resources.md).
![file upload](res/sandbox_upload_file_selector.png "File Upload")
## Object storage
To use the Object storage, you can upload a file via REST-Interface and access it by the key you get provided in the response. If you want to upload your file:
## Headless file storage
To use the headless object storage, you can upload a file via REST-Interface or curl. The json response message provides you the destination url.
**Upload Example**
```python
@ -22,8 +19,16 @@ To use the Object storage, you can upload a file via REST-Interface and access i
files = {'fileUpload': (filename, open(filename, 'rb'),'text/csv')}
r = requests.post(url, files=files)
```
**Example JSON response**
```json
{
"PublicUrl": "https://storage.sandbox.iuk.hdm-stuttgart.de/upload/a1236b2b-49bf-4047-a536-20dab15b7777/untitled.txt",
"Size": 11,
"Expiration": "2023-10-04T00:00:00Z"
}
```
**Usage Example**
**Import Example**
```python
import pandas as pd
@ -31,6 +36,7 @@ To use the Object storage, you can upload a file via REST-Interface and access i
df = pd.read_csv(url)
```
## Git LFS
The following solution we highly recomment only for users which are familiar with git command line tools! Git Large File Storage (LFS). An open source Git extension for versioning large files. Git LFS replaces large files (audio, sample, datasets, videos) by a text pointer inside git. The files get stored on our gitea Server.
For further information visit [Git LFS](https://git-lfs.com/).

48
src/sandbox/dev_env.md
View File

@ -3,29 +3,30 @@
![sandbox](https://uptime.monitoring.iuk.hdm-stuttgart.de/api/badge/1/status)
## Introduction
use cases, limitations + t&c
The **Sandbox Development Environment** enables students and lecturers to create and work with interactive case studies. It also provides a development environment for researchers and advanced programmers.
The Sandbox is available at: https://sandbox.iuk.hdm-stuttgart.de
Please carefully handle the resources and follow the fair use principle and as well as the German laws.
## Getting Started
Within the sandbox, the different disciplines inside the IKID project can provide tasks to be worked on by the respective student groups. Both text-based tasks and programmatic tasks can be provided and processed. For example, Markdown files can be created for editing textual tasks. These files can be converted from the source form (unformatted) to the target form (formatted) using a simple syntax.
Currently, it is planned for the technical lectures that the students get first in touch with the programming language Python. Therefore, the Sandbox platform was created, in which experiments with Python can be carried out. But if needed, it is possible to add more supported languages in the future.
[Sandbox](https://sandbox.iuk.hdm-stuttgart.de/)
(only accessible from the HdM-Network)
Within the sandbox, the different disciplines of the IKID project can provide tasks to be worked on by the respective student groups. Both text-based tasks and programmatic tasks can be processed. For example, Markdown files can be created for editing textual tasks.
Currently, it is planned for the technical lectures that students get in touch with the programming language Python for the first time. Therefore, the Sandbox platform was created, in which experiments with Python can be carried out.
1. **Sign in**, use your **HdM Credentials**
1. **Sign in**, use your **HdM Credentials** at [Sandbox](https://sandbox.iuk.hdm-stuttgart.de/)
2. Select the image you want to start (two options)
1. **Datascience environment**
2. **GPU environment** (choose only if you realy need the graphic card, otherwise you steal resources from those who need them)
3. Create or upload a .ipynb file to start with
2. **Datascience GPU environment** (choose only if you realy need the graphic card, otherwise you block resources from those who need them)
3. Create or upload a .ipynb file:
1. . **create a empty .ipynb file:**
![sandbox launcher](res/sandbox_launcher.png "Sandbox Launcher")
2. **upload a existing .ipynb file:**
![sandbox upload file](res/sandbox_upload_file_selector.png "Sandbox upload file")
4. **open** the file from the filebrowser & start working!
![sandbox .ipynb file](res/sandbox_ipynb_example.png "Sandbox Notebook").
5. **After you finished your work dont forget to shutdown your server!** Therfore you should shutdown your server to release server resources. **Select File, Hub Control Panel**
5. **After you finished your work dont forget to shutdown your server!** Therefore, you should shutdown your server to release server resources. **Select File, Hub Control Panel**
![file menue](res/sandbox_file_menu.png "File menue")
7. Select **Stop Server**
![stop server](res/sandbox_stop_server.png "Stop Server")
@ -33,22 +34,33 @@ Currently, it is planned for the technical lectures that the students get first
![logout](res/sandbox_logout.png)
## Technical Overview
Which python packages are installed, How can I install a python package?
### Environments
The Sandbox provides multiple scientific environments. If any additional packages or libraries are need, please open an issue on our [GIT](https://git.sandbox.iuk.hdm-stuttgart.de/).
#### Datascience environment
* Available Data Science image is based on [Official Data Science Image](https://hub.docker.com/r/jupyter/datascience-notebook/tags/)
* most common data analysis library's included for Julia, Python, R
* Available Data Science image is based on [Data Science Container](https://git.sandbox.iuk.hdm-stuttgart.de/grosse/-/packages/container/jupyterlab-datascience/latest)
* most common data analysis library's included for Julia, Python, R are available
#### GPU environment
* Available GPU image is based on [Official GPU Image](https://hub.docker.com/r/cschranz/gpu-jupyter)
* support added for the NVIDIA GPU A100 calculations based on python most common GPU-able libraries Tensorflow, PyTorch and Keras.
Please open an [issue](https://git.sandbox.iuk.hdm-stuttgart.de/grosse/jupyterlab-datascience/issues/new) if any additional packages are needed or issues occurred.
#### Datascience GPU environment
* Available GPU image is based on [Datascience GPU Container](https://git.sandbox.iuk.hdm-stuttgart.de/grosse/-/packages/container/jupyterlab-datascience-gpu/latest)
* support added for the NVIDIA GPU A100 computations based on python most common GPU-able libraries like Tensorflow, PyTorch and Keras.
Please open an [issue](https://git.sandbox.iuk.hdm-stuttgart.de/grosse/jupyterlab-datascience-gpu/issues/new) if any additional packages are needed or issues occurred.
### Resources
Each instance has following resource limits:
- maximum 2 physical CPUs, guranteed 0.5 CPUs
- maximum 10GB DDR memory, guranteed 1GB
- maximum of 1GB HDD
For GPU enabled environments, 40GB shared (time sliced) GPU memory is availble. For additional information, please see the [official nvidia documentation](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/index.html).
### Limitations
### Limitations
Please carefully follow the terms and conditions at [Sandbox](https://sandbox.iuk.hdm-stuttgart.de) website. The HDD free space can not be extended. The Sandbox should only be used for short running GPU tasks. For longer running trainings, please use the [Training Environment](sandbox/training.md). After GPU usage please stop your kernel/instance or free the blocked GPU resources manually. After 30 minutes of inactivity, the instanced will automatically removed. The HDD space is persistent, but will be deleted after 6 months.

4
src/sandbox/training.md
View File

@ -1,6 +1,4 @@
# Training Environment
# Training
Coming Soon ...
Currently under heavy development, coming soon....stayed tuned!