Webius Officium ─ NiMBUS deBlanc

Name: NiMBUS deBlanc (official) | 구운몽 (korean)

Model: Raspberry Pi 3 Model B Raspbierry Pi Model B

Storages — Storage in details

• 240G — /var/www partition
• 500G — ~/storage partition

Mounted on     Type  Size  Used Avail Use%
/              ext4   15G  2.0G   12G  15%
/var/www       ext4  220G  467M  208G   1%
/home/www-data ext4  458G  313G  122G  72%

— Júne's home network

• Wired (NiMBUS-enx001)
• Wireless Connection (NiMBUS-wlx001)
• Wired (NiMBUS-wlx002)

Web publishing and web resources including WebDAV

admin Júne's PiON Gateway — WebDAV and Documents Archives

admin Júne's Cloud Platform — Gateway for all cloud services

admin Júne's Wiki — What I Know Is … Here

Nginx Configuration Showing lively on OWL for NiMBUS deBlanc

• adjust & optimize value of php-fpm – /etc/php/7.x/fpm/pool.d/www.conf _{— Updated on 2020/07/25 21:40}
• proxy_pass for transmission web with bit.meson.in, tor.meson.in & gen.meson.in _{— Updated on 2020/07/25 04:15}
• Fresh installation _{— Updated on 2020/07/25 04:13}
• Change backup rsync to another partition _{— Updated on 2020/06/27 04:01}
• Change web root of eigen.ml, dav.meson.in & pdf.meson.in _{— Updated on 2020/06/27 04:00}
• Plugin Wireless LAN & Wired Giga LAN _{— Updated on 2020/02/22 03:32}
• Fresh installation _{— Updated on 2020/02/11 02:16}
• configure access.log and error.log separately _{— Updated on 2018/09/20 11:12}
• create server block for [ dav.meson.in ] _{— Updated on 2018/08/14 15:35}
• Optimize Nginx configuration for Dokuwiki reference content
• create server block for [ lib.meson.one ] library project _{— Updated on 2018/04/11 18:22}
• Activate gzip module in Nginx _{— Updated on 2018/02/27 06:10}
• Configured HTTP/2 for all sites _{— Updated on 2018/02/15 19:45}
• Finished to publish front page of pi.meson.one and cloud.meson.in
• Add USB Wireless (802.11n) and assigned _{— Updated on 2018/01/31 19:47}

• rsync with cloud service such as Box or Dropbox

• Check security for WebDAV access and configuration

Edit /boot/config.txt

dtoverlay=disable-wifi
dtoverlay=disable-bt

Or add configuration string in config.txt

echo "dtoverlay=pi3-disable-wifi" | sudo tee -a /boot/config.txt
echo "dtoverlay=pi3-disable-bt" | sudo tee -a /boot/config.txt

Disable systemd service that initializez Bluetooth Modems connected by UART.

sudo systemctl disable hciuart.service

Default (initial) configuration for Wireless is stored in /etc/wpa_supplicant/wpa_supplicant.conf

If another wireless device is wlan1 then copy as wpa_supplicant-wlan1.conf and edit

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=wheel
network={
    ssid="example"
    scan_ssid=1
    key_mgmt=WPA-EAP WPA-PSK IEEE8021X NONE
    pairwise=CCMP TKIP
    group=CCMP TKIP WEP104 WEP40
    psk="very secret passphrase"
    eap=TTLS PEAP TLS
    identity="user@example.com"
    password="foobar"
    ca_cert="/etc/cert/ca.pem"
    client_cert="/etc/cert/user.pem"
    private_key="/etc/cert/user.prv"
    private_key_passwd="password"
    phase1="peaplabel=0"
    ca_cert2="/etc/cert/ca2.pem"
    client_cert2="/etc/cer/user.pem"
    private_key2="/etc/cer/user.prv"
    private_key2_passwd="password"
}

sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

Login with Superuser root and clone required packages. Target directory is /root/letsencrypt. Target directory could be wherever you want, which you should be careful about security and permission. Therefore, /root directory is a recommended whereas.

  sudo su 
  git clone https://github.com/letsencrypt/letsencrypt /root/letsencrypt 

Go to target directory to run the script

  cd /root/letsencrypt 
  ./certbot-auto certonly --webroot --rsa-key-size 4096 -w /var/www/YOUR_ROOT_DIR/ -d YOUR.DOMAIN.COM 

YOUR_ROOT_DIR is root directory for the service of YOUR.DOMAIN.COM. –rsa-key-size could be altered with another bit such as 2048 or less but 4096 is better choice for better security.

After reviewing the successful messages, then default location of certificates is /etc/letsencrypt/live/YOUR.DOMAIN.COM/. Usual error has something to do with not loading any pages in http protocol. So you make sure even simple page (landing page) in YOUR_ROOT_DIR properly.

MAKE SURE the renewal of certificates every 90 days

For renewal certificates, run certbot-auto and follow the proceeds.

./certbot-auto --domains YOUR.DOMAIN.COM

Renew ALL certificates from Let's Encrypt, run

./certbot-auto renew

Delete certificates from Let's Encrypt, run

./certbot-auto delete

 sudo apt update 
 sudo apt install certbot python-certbot-nginx 

command is same as certbot-auto by root privilege.

 certbot certonly --webroot -w /var/www/mydomain -d www.mydomain.com 

 certbot renew

 certbot delete --cert-name delete.mydomain.com 

Check renewal executes without error,

 sudo certbot renew --dry-run 

If no error, edit service unit file typically stored in /etc/systemd/system/. Edit /etc/systemd/system/certbot-renewal.service

[Unit]
Description=Certbot Renewal
 
[Service]
ExecStart=/usr/bin/certbot renew --post-hook "systemctl restart nginx.service"

Which restarts web service after renewing certificate(s).

Modify /etc/systemd/system/certbot-renewal.timer to adjust timer for certbot renewal.

[Unit]
Description=Timer for Certbot Renewal
 
[Timer]
OnBootSec=300
OnUnitActiveSec=2w
 
[Install]
WantedBy=multi-user.target

The configuration below will activate the service biweekly, and 300 seconds after boot-up.

To start the timer

  sudo systemctl start certbot-renewal.timer

To enable the timer to be started on boot-up

  sudo systemctl enable certbot-renewal.timer

To show status information for the timer

  systemctl status certbot-renewal.timer

To show journal entries for the timer

  journalctl -u certbot-renewal.service

  openssl dhparam -out /..path../dhparma.pem 2048 

4096 bit requires (approximately) 7 times CPU resources than 2048

This configuration is for Nginx Web Server. edit /etc/nginx/sites-available/default

    # SSL configuration
 
     listen 443 ssl default_server;
     listen [::]:443 ssl default_server;  
    # default_server should be deleted for multi-casting domains 
 
     ssl_certificate /etc/letsencrypt/live/... (path) .../fullchain.pem;
     ssl_certificate_key /etc/letsencrypt/live/... (path) .../privkey.pem;
     ssl_session_timeout 5m;
     ssl_session_cache shared:SSL:20m;
 
    # Diffie-Hellman parameter for DHE ciphersuites
     ssl_dhparam /... (path) .../dhparam.pem;
 
    # Protocols and Ciphers
     ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
     ssl_prefer_server_ciphers on;
    # last two ciphersuites only added because of Android 4.3
     ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA';
 
    # Add headers to serve security related headers
     add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;";
     add_header X-Content-Type-Options nosniff;
     add_header X-Frame-Options "SAMEORIGIN";
     add_header X-XSS-Protection "1; mode=block";
     add_header X-Robots-Tag none;
     add_header X-Download-Options noopen;
     add_header X-Permitted-Cross-Domain-Policies none;
 
    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
     ssl_stapling on;
     ssl_stapling_verify on;
 
    # verify chain of trust of OCSP response using Root CA and Intermediate certs
     ssl_trusted_certificate /etc/letsencrypt/live/... (path) .../chain.pem;

Nginx live configuration

WebDAV is web protocol based service to open directory and web resources via various ways. To enable WebDAV or Open directory in Nginx. Append the following code inside Server { …} line.

  location /webdav {
           alias     /home/user/WebDAV;
           client_body_temp_path /home/user/WebDAV/.temp;
 
           dav_methods     PUT DELETE MKCOL COPY MOVE;
           dav_ext_methods   PROPFIND OPTIONS;
 
           create_full_put_path  on;
           dav_access    user:rw group:rw all:r;
           autoindex     on;
           autoindex_format     html;     # OPTION: HTML | XML | JSON 
           auth_basic "TITLE FOR YOUR WEBDAV";
           auth_basic_user_file /home/user/WebDAV/.htpasswd;
 
           #fancyindex on; 
           #fancyindex_exact_size off; 
 
         }

  #here you can specify various directories that respond as DAV.
  location /ergo-repo/ {
    root      /var/dav;
    client_body_temp_path /var/dav/temp;
    dav_methods     PUT DELETE MKCOL COPY MOVE;
    dav_ext_methods   PROPFIND OPTIONS;
    create_full_put_path  on;
    dav_access    user:rw group:rw all:rw;
    autoindex     on;
    #below you can specify the access restrictions. In this case, only people on the 141.142 network
    #can write/delete/etc. Everyone else can view.
    limit_except GET PROPFIND OPTIONS{
      allow 141.142.0.0/16;
      deny  all;
    }
    allow all;
  }
  #this is an example of a password restricted repository
  location /password-repo/ {
    root      /var/dav;
    client_body_temp_path /var/dav/temp;
    dav_methods     PUT DELETE MKCOL COPY MOVE;
    dav_ext_methods   PROPFIND OPTIONS;
    create_full_put_path  on;
    dav_access    user:rw group:rw all:rw;
    autoindex     on;
    auth_basic "restricted";
    auth_basic_user_file /etc/nginx/htpasswd;
  }

Fancyindex is an optional part if you have installed nginx-extra or nginx-full.

When fancyindex is on, comment out autoindex directive.

#  autoindex on;
 
   fancyindex on; 
   fancyindex_exact_size off; 

.htpasswd file is authentication file for user to access correctly. To generate this file, install htpasswd.

 sudo apt-get install apache2-utils  

If you don’t have an .htpasswd or create new one, use the -c option to generate the file with the first user. It will prompt you for a password and encrypt it for you.

  htpasswd -c /..path../ .. /../.htpasswd firstuser

To add another user for this,

  htpasswd /..path../ .. /../.htpasswd seconduser

Delete user. If the username exists in the specified .htpasswd file, it will be deleted.

  htpasswd -D /..path../ .. /../.htpasswd firstuser 

Access https://YOUR.DOMAINS.COM/WebDAV in browser and check whether prompt dialogue pops up.

Using rsync, backup and archive web resource into another location.

  30 5 * * 3 /usr/bin/rsync -Aax /var/www/ /.. (backup path..) ../web_backup_`date +"%Y%m%d"`/ 

Assume web root directory is /var/www/ and to create backup folder with date,
Append crontab with root privilege - sudo crontab -e.
30 5 * * 3 means run script at 05:30 (AM) every Wednesday (Mon:1 … Sun:7 or 0)

Fancyindex modules in Nginx

  sudo apt-get install nginx-extras 

Add fancyindex directives in virtual host configuration files.

  location / {
    fancyindex on;
    fancyindex_exact_size off
  }

Insert following lines to let Fancyindex work in the web directory.

   fancyindex on;
   fancyindex_localtime on;
   fancyindex_exact_size off;
  # Specify the path to the header.html and foother.html files (server-wise)
   fancyindex_header "/Nginx-Fancyindex-Theme/header.html";
   fancyindex_footer "/Nginx-Fancyindex-Theme/footer.html";
  # Ignored files will not show up in the directory listing, but will still be public.
   fancyindex_ignore "examplefile.html";
  # Making sure folder where these files are do not show up in the listing.
   fancyindex_ignore "Nginx-Fancyindex-Theme";
  # Maximum file name length in bytes, change as you like.
   fancyindex_name_length 255;

Adding in /etc/nginx/sites-available/YOUR-WEB-CONFIG

  error_page 404 /custom_404.html;
        location = /custom_404.html {
                root /var/www/html;
                internal;
        }

For 50x page,

  error_page 500 502 503 504 /custom_50x.html;
        location = /custom_50x.html {
                root /var/www/html;
                internal;
        }

For test purpose, adding a dummy FastCGI pass with following lines.

    location /testing {
                fastcgi_pass unix:/does/not/exist;
        }

• dav_ext_methods directive failed
• fancyindex directive failed

This problem is due to not loading proper module in Nginx configuration.
Edit /etc/nginx/nginx.conf

    user www-data;
    worker_processes auto;
 
    pid /run/nginx.pid;
 
  include /etc/nginx/modules-enabled/*.conf;
 
    events {
      worker_connections 12288;
    } 

If this does not work, please check modules are located correctly or not in /etc/nginx/modules-enabled. If not, run sudo apt-get install nginx-extra.

sudo apt-get update
sudo apt-get install apache2-utils

Generating (creating) password file to enable authentication

sudo htpasswd -c /etc/nginx/.htpasswd username 

Appending user in existing password file

sudo htpasswd /etc/nginx/.htpasswd another_user

Edit /etc/nginx/sites-available/default

listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;

Edit /etc/nginx/nginx.conf

insert following line after

ssl_prefer_server_ciphers on;

ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;

HTTTP/2 online test: HTTP2 Pro

For more information: How To Set Up Nginx with HTTP/2 Support on Ubuntu 16.04

Theoretical background: Google Web Fundementals

Create /web.root/expires.conf site dedicated configuration file.

# Expire rules for static content
 
# cache.appcache, your document html and data
location ~* \.(?:manifest|appcache|html?|xml|json)$ {
  expires -1;
  # access_log logs/static.log; # I don't usually include a static log
}
 
# Feed
location ~* \.(?:rss|atom)$ {
  expires 1h;
  add_header Cache-Control "public";
}
 
# Media: images, icons, video, audio, HTC
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
  expires 1M;
  access_log off;
  add_header Cache-Control "public";
}
 
# CSS and Javascript
location ~* \.(?:css|js)$ {
  expires 1y;
  access_log off;
  add_header Cache-Control "public";
}

include conf in Nginx site configuration

server {
    # Note that it's listening on port 9000
    listen 9000 default_server;
    root /web.root/;
    index index.html index.htm;
 
    server_name example.com www.example.com;
 
    charset utf-8;
    include /web.root/basic.conf;
 
    location / {
        try_files $uri $uri/ =404;
    }
}

Nginx Manual

edit /etc/nginx/nginx.conf add following lines or uncomment out

gzip on;
gzip_disable "msie6";
 
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

Verify working using cURL with Accept-Encoding

  curl -H "Accept-Encoding: gzip" -I https://web.url/uri/object.css 

data directory

wiki/to/path$ sudo chmod -R g=rwX,u=rwX,o=rX data/
wiki/to/path$ sudo chown -R www-data:www-data data/

everything below the data directory

wiki/to/path/data$ sudo chmod 2775 {attic,cache,index,locks,media,meta,pages,tmp}
wiki/to/path/data$ sudo chown www-data:www-data {attic,cache,index,locks,media,meta,pages,tmp}

For newly created directories, it might require setgid bit in order to fully retain correct permissions after setting up the existing ones.

Problem: Error during sudo apt update

Symptom

apt-get: symbol lookup error: /usr/lib/arm-linux-gnueabihf/libapt-pkg.so.4.12: undefined symbol: 

Solution: reinstall package

# Download the current version of libapt-pkg4.12
wget http://mirrordirector.raspbian.org/raspbian/pool/main/a/apt/libapt-pkg4.12_0.9.7.9+rpi1+deb7u7_armhf.deb
 
# Install it
sudo dpkg -i libapt-pkg4.12_0.9.7.9+rpi1+deb7u7_armhf.deb

Problem: Error found in log /var/log/nginx/xxx_error.log,

  upstream timed out (110: Connection timed out) while reading response header from upstream 

Symptom: No symptom while running website. Only found in error log.

Solution: Solution #1

Problem: Error found in log /var/log/kern.log,

  Under-voltage detected! (0x00050005)

Symptom: No symptom while running website. Only found in error log.

Solution: