Update readme

README2.md

@@ -11,7 +11,7 @@ Assign users to custom roles, search in inventory or provision new virtual machi
 * Version : Alpha 1.0
 * Status: Dev
 * Object :   Massive LXC CT / KVM deployment, management and viewer system for Proxmox clusters.
-* Proxmox version supported: 3.x/4.x/5/x
+* Proxmox version supported: 3.x/4.x/5.x
 * Information : 
 This project is currently in active development. 
 You shouldn't use in production mode or... use at your risks !
@@ -24,7 +24,7 @@ You shouldn't use in production mode or... use at your risks !
 - Infrastructure historic
 - Instance management (stop/start/restart...)
 - Search system by VM-name, mac address ...
-- Proxmox crawler (Get and store information)
+- Proxmox crawler (Getting cluster information)
 - Security : Encipher the critical data (cluster access)
 - LDAP authentication for web interface
 - Group & cluster viewing in web interface

doc/md/01-prerequisites.md

@@ -9,7 +9,7 @@ You have to the possibility to access on the Proxmox API with an Administrative
 Setting up Proxmox policies is out of scope for this tutorial.  
 Report you to the official Proxmox documentation.  
 
-## Recent environment for installing this project
+## Environment
 This project need an recent environment to work correctly.  
 Typically, in this documentation, we'll use an Debian 9 environment.  
 You can use an other distribution, but just check if theses software are available:
@@ -21,8 +21,8 @@ You can use an other distribution, but just check if theses software are availab
 
 This project is separate in two parts: Frontend and backend.  
 Backend is writing in python and frontend in PHP/HTML/CSS/JS.
-You have the possibility to use these two parts on different machines, 
-but du performance you should use the same (low network latency).  
+You have the possibility to use these two parts on different machines,
+but due to performance you should use the same (low network latency).  
 In more, the API authentication system is currently not implemented and this architecture can expose you
 to security issues.
 
@@ -33,6 +33,6 @@ But on a large production system, it can need more resources, especially for the
 If your Proxmox infrastructure is very large, the data generated can be really important,
 and cause a slowly working if your hardware is too low.
 
-In some case, it can be necessary to use an independent MongoDB machine or cluster.
+In some case, it can be necessary to use an independent MongoDB server or cluster.
 
-[Readme](../../README2.md) <-- Previous | Next --> [Setup - Backend](doc/md/02-backend.md)
+[Readme](../../README2.md) <-- Previous | Next --> [Setup - Backend](02-backend.md)

doc/md/02-backend.md

@@ -0,0 +1,68 @@
+# Backend
+
+
+### Databases
+The backend need two database servers: MongoDB and Redis.  
+MongoDB is the main data storage and redis is used like a cache system.  
+HyperProxmox can run with default configuration, but you should setup redis to work in full memory, and the same
+for the MongoDB indexes.
+ 
+``` bash
+apt-get install mongodb nginx redis-server
+```
+
+#### Setup Redis
+
+Redis-server can work with a small memory-cache dedicated.
+
+vi /etc/redis/redis.conf
+```bash
+# Networking
+bind 127.0.0.1
+port 6379
+tcp-keepalive 60
+ 
+# Maximum memory
+maxmemory 256mb
+maxmemory-policy allkeys-lru
+ 
+# Disable disk persistence
+appendonly no
+save ""
+```
+
+#### Setup MongoDB
+
+vi /etc/mongodb.conf
+```bash
+bind_ip = 127.0.0.1
+port = 27017
+```
+
+## Install python
+
+``` 
+apt-get install python3-redis python3-netaddr python3-pip python3-webpy python3-requests
+pip3 install pymongo
+```
+
+## Install sources files
+``` bash
+apt-get install git
+
+useradd hyperproxmox
+cd /opt/ && git clone https://github.com/ThomasGsp/HyperProxmox.git
+
+# Set hyperproxmox
+chown hyperproxmox: -R /opt/HyperProxmox
+chmod 760 -R /opt/HyperProxmox
+
+# Log dir (you can change it)
+mkdir /var/log/hyperproxmox/
+chown hyperproxmox: /var/log/hyperproxmox/
+
+#Rm demo keys
+rm /opt/HyperProxmox/code/scripts/main/private/keys/Ragnarok.p*
+```
+
+[Prerequisites](01-prerequisites.md) <-- Previous | Next --> [Setup - Backend](03-frontend.md)

doc/md/03-frontend.md

@@ -0,0 +1,80 @@
+# Frontend
+
+### Web server
+The frontend is writing in php/html/css/js.  
+You can use every web servers that support theses technologies.  
+For this installation, we'll setting up nginx with php-fpm.
+
+``` bash
+apt-get install nginx php-fpm php-curl php-json
+```
+
+#### Nginx
+vi /etc/nginx/sites-available/hyperproxmox.conf
+``` bash
+server {
+   listen *:443 ssl;
+   server_name youdomain.name;
+   root /var/www/hyperproxmox;
+   
+   ssl on;
+   ssl_certificate /etc/nginx/ssl/nginx.crt;
+   ssl_certificate_key /etc/nginx/ssl/nginx.key;
+   ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
+   ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
+   ssl_prefer_server_ciphers on;
+   add_header Strict-Transport-Security "max-age=86400";
+    location ~ \.php$ {
+        include snippets/fastcgi-php.conf;
+        fastcgi_pass unix:/var/run/php-www.sock;
+    }
+}
+
+```
+
+In this example, i used the pre-generates nginx key: "/etc/nginx/ssl/nginx.key"  
+You should change this part.  
+
+#### Php-fpm
+vi /etc/php/7.0/fpm/pool.d/www.conf
+``` bash
+[www]
+
+user                         = www-data
+group                        = www-data
+
+listen                       = /var/run/php-www.sock
+listen.owner                 = www-data
+listen.group                 = www-data
+listen.mode                  = 0660
+
+pm                           = dynamic
+pm.start_servers             = 5
+pm.min_spare_servers         = 5
+pm.max_spare_servers         = 35
+pm.max_children              = 50
+
+pm.max_requests              = 200
+
+pm.status_path               = /fpm-status
+ping.path                    = /ping
+ping.response                = pong
+
+request_slowlog_timeout = 0
+
+request_terminate_timeout    = 0
+catch_workers_output = yes
+
+```
+
+#### Web application
+``` bash
+# set www dir
+mkdir /var/www/hyperproxmox
+cp -R /opt/HyperProxmox/code/web/www/* /var/www/hyperproxmox/
+chown www-data: -R /var/www/hyperproxmox
+# No www-data write (useless)
+chmod 550 -R /var/www/hyperproxmox
+```
+
+[Setup - Backend](02-backend.md) <-- Previous | Next --> [Setup - Configs](04-configs.md)

doc/md/04-configs.md

@@ -0,0 +1,84 @@
+# Configs
+
+Configurations are managed by single point:
+
+vi /opt/HyperProxmox/code/scripts/main/private/conf/config
+``` bash
+[system]
+; System configurations
+user: hyperproxmox
+
+; If not exist at startup, the key will be auto-generate.
+key_pvt: private/keys/Ragnarok.pvt.key
+key_pub: private/keys/Ragnarok.pub.key
+
+admin_mail: tlams@localhost
+
+[web]
+user: www-data
+
+[api]:
+user: hyperproxmox
+
+[databases]
+; Databases configurations
+; NOSQL databases, should use a password
+mongodb_user:
+mongodb_password:
+mongodb_ip: 127.0.0.1
+mongodb_port: 27017
+
+redis_user:
+redis_password:
+redis_ip: 127.0.0.1
+redis_port: 6379
+
+[deploy]
+; Maximum concurrent deployment
+; A high value can overcharge your physicals servers
+concurrencydeploy: 2
+
+; Delay between two deployment round
+; If your infrastructure isn't very large, you should'nt reduce this delay.
+; A low delay can overcharge your physicals servers
+delayrounddeploy: 15
+
+[walker]
+; Delay in seconds between to crawl (update)
+walker: 300
+
+; Lock file -- prevent concurrent crawling
+walker_lock: /tmp/hyperproxmoxwalker.lock
+
+; Set an unique ID (change comment part)
+uid = False
+
+[logger]
+; logs level    1: "INFO",  2: "WARNING", 3: "ERROR", 4: "CRITICAL",  5: "DEBUG"
+logs_level = 5
+
+; Limit IO write, if debug level is active, this value is overwrite to 0
+bulk_write = 1
+
+; Buffer size
+bulk_size = 1000
+
+; log output
+logs_dir = /var/log/hyperproxmox/
+```
+
+#### Purge system
+You should setup an cron to purge old data.
+``` bash
+RET=4 # older than the current date less this delay in days
+DATETIMESTAMP=$(($(date +%s)-$((86400*$RET))))
+curl -H -XPOST -d '{ "action": "purge", "type":"strict", "date": $DATETIMESTAMP }'  localhost:8080/api/v1/administration/purge
+```
+* action: actiontype (only purge is currently available)
+* type: purge type (strict = all data before this date)
+* date: delete data before this date - in seconds(timestamp)
+
+
+
+
+[Readme](03-frontend.md) <-- Previous | Next --> [Usage - First start](05-first_start.md)