Featured

Building a Professional RestFul API in Python – Part 1

Before starting this series of posts, I wanted to thank the comments that you have been leaving on my blog, as they encourage me to continue posting things on this site. If you have read the title of this post you will notice that it is not very related to the publications that I have done so far, but yes, I am dedicated to web programming ūüôā

As I go through the project, all the code will be available on my GitHub. The code that I’m going to show in this post is based on the commit: 6917f480f74a3e03c15c37fdbca2ca0f09548ac

https://github.com/bounteous/PackTrack-Api

To not mix the versions of the dependencies of our project, let’s create a new python environment following the instructions of the following link link

We are gonna use Flask, it’s gonna be useful for creating the routes into our project, install that dependence by following the steps from the link below link

First of all, let’s check the index.py file. That file is the responsible to declare not only the routes for the API, it’s also the place for locating the different methods for the URLs

This may seem overwhelming at first, but you will understand it in 5 minutes, we will follow the code line by line

First of all, we declare the environment of the app, which will be the relative path where flask is installed in our python env, created at the beginning. Then, Flask and our custom LIBs are imported. Next, we do the same importing our routes, the data models and the custom module. So before continue we need to understand the main lib until the moment, which is the file functions.py located inside utils folder.

As you can observe all the functions are located inside a “try catch” block, that’s because we are building a professional API ūüėČ In case some of the code returns an error during the execution, the endpoint is not gonna return the technical error to the client, it’s gonna return the human error. That logic can be found on the function “setModuleError and setModuleSuccess” which uses the model “_rest” imported at the beginning of the file (check the image below)

The “setStatus” function is used for checking that the request status that is returned to the client contents a valid status code for being interpreted on the client side (just for verifying). And the other one “flaskResp” is the responsible of creating a valid return construction for being send from the Flask module in our app

Volviendo de nuevo a nuestro archivo functions.py, podemos observar en cada Excepci√≥n del c√≥digo, c√≥mo se crea una instancia del nuevo “error de objeto de respuesta” y est√° listo para devolver los datos del resultado a la ruta ubicada en el archivo index.py

As you can see on the last image, we need to construct the “setFlaskResponse” object explained before for sending the response and the status code assigned if the result is success or not.

On that image I made a first request from the left terminal, then I introduced a syntax error on the code and then I made again the same request on the right terminal. As you can observe from the terminal on the top, you can read the technical error and the human error response sent back to the user.

The next post will be for the signup ūüôā

zRAM on hot using a udev rule

This post explains how to configure zRAM on a Linux system running systemd using udev rules. In the case of wanting to know internally the operation of zRAM visit: Gentoo Wiki

Basically the things we have to keep in mind are the following. Each zRAM device is limited to being managed by 1 CPU core, so we will have to create a device for each core in case we want to make the most of a CPU with multiple cores.

This technology, which is available since version 3.15 of the Linux kernel, can not only be used to store the well-known swap that is typically stored on the hard disk, but can also be used as a reserved temporary storage device. This is a good practice for temporary folders of the system, such as temporary compilation folders, volatile data (/tmp), web cache, etc …

Before making the new zram devices permanent in our system, we will load the zram module already included in the kernel of our distribution (in case you have a custom kernel visit the link to the Gentoo wiki).

My CPU has 4 cores, so I’m going to create the same blocks in RAM. Edit: /etc/modprobe.d/zram.conf

options zram num_devices=4

Create the rules. Edit: /etc/udev/rules.d/99-zram.rules

KERNEL==”zram0″, ATTR{disksize}=”512M” RUN=”/usr/bin/mkswap /dev/zram0″, TAG+=”systemd”
KERNEL==”zram1″, ATTR{disksize}=”512M” RUN=”/usr/bin/mkswap /dev/zram1″, TAG+=”systemd”
KERNEL==”zram2″, ATTR{disksize}=”512M” RUN=”/usr/bin/mkswap /dev/zram2″, TAG+=”systemd”
KERNEL==”zram3″, ATTR{disksize}=”512M” RUN=”/usr/bin/mkswap /dev/zram3″, TAG+=”systemd”

Load the module, you will se the new devices created on /dev/zram*

modprobe zram

Now you can swap devices, do the same on the rest of devices 1, 2, 3

swapon /dev/zram0

To see if we have configured the swap in zRAM correctly, we execute:

free -h

For loading the module on systemd boot automatically:

/etc/modules-load.d/zram.conf

Finally add to the end of the file /etc/fstab

/dev/zram0              none            swap            defaults        0 0
/dev/zram1              none            swap            defaults        0 0
/dev/zram2              none            swap            defaults        0 0
/dev/zram3              none            swap            defaults        0 0

Be good guys!

Archlinux Install – UEFI+LVM+LUKS+SystemdBoot

I’m installing Archlinux on a USB from my laptop that already has an Archlinux installed. The process that I will perform to install Archlinux on the new storage device, is the same as installing the distribution from the official ISO of Archlinux, in the end it is about having a shell to install it from there.

First of all check if your device has booted up from UEFI mode.

ls /sys/firmware/efi

Next we select the disk in which it stores the main installation of our system.

fdisk -l

In my case the selected disk is /dev/sda

gdisk /dev/sda

Command (? for help): o
This option deletes all partitions and creates a new protective MBR.
Proceed? (Y/N): Y

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-60437458, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-60437458, default = 60437458) or {+-}size{KMGTP}: +512MiB
Current type is ‘Linux filesystem’
Hex code or GUID (L to show codes, Enter = 8300): ef00
Changed type of partition to ‘EFI System’

Command (? for help): n
Partition number (2-128, default 2):
First sector (34-60437458, default = 1050624) or {+-}size{KMGTP}:
Last sector (1050624-60437458, default = 60437458) or {+-}size{KMGTP}:
Current type is ‘Linux filesystem’
Hex code or GUID (L to show codes, Enter = 8300): 8e00
Changed type of partition to ‘Linux LVM’

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): Y
OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.

We are going to format the boot partition in fat32

mkfs.fat -F32 /dev/sda1

Before creating the volumes for the partition of type LVM, we need to encrypt your partition beforehand. In this way we do not save having to encrypt each logical volume, since it is vastly inefficient.

cryptsetup luksFormat /dev/sda2

To mount the new partition now with luks we need to decrypt it in advance.

cryptsetup open –type luks /dev/sda2 lvm

For creating a physical volume:

pvcreate /dev/mapper/lvm

Create your first volume group and logical volumes:

vgcreate volume /dev/mapper/lvm

lvcreate -L2G volume -n swap

lvcreate -l 100%FREE volume -n root

We have already used 100% space from /dev/sda2, which is encrypted and with a configured LVM system. It’s time to choose the FS. In my case it will be Ext4 for the root partition.

mkfs.ext4 /dev/mapper/volume-root

Swap setup:

mkswap /dev/mapper/volume-swap

We are going to download from the official Archlinux repositories a basic system that we will assemble and access through chroot.

mount /dev/mapper/volume-root /mnt

mkdir /mnt/boot

mount /dev/sda1 /mnt/boot

swapon /dev/mapper/volume-swap

pacstrap /mnt base base-devel

Now we are going to store the currently mounted units in /etc/fstab so that the location of the partitions can be located in the next system start so that the partitions are mounted. If the following command is not found try installing (pacman -Syy arch-install-scripts)

genfstab -p /mnt >> /mnt/etc/fstab

To access the new system:

arch-chroot /mnt

Setup your location:

ln -sf /usr/share/zoneinfo/Europe/Madrid /etc/localtime

hwclock –systohc –utc

Configure the root user.

passwd root

To set the type of coding in your system, remove the comment from the line en_US.UTF-8:

nano -w /etc/locale.gen

locale-gen

Create the file /etc/hostname and set your machine name, that’s a crucial step ūüėČ

echo “Tux” > /etc/hostname

Now we need to add new modules to our initramfs (follow the order).

nano -w /etc/mkinitcpio.conf

HOOKS=(base udev autodetect modconf block keyboard encrypt lvm2 filesystems fsck)

mkinitcpio -p linux

Regenerate it and store the new initramfs on /boot:

We will install the integrated boot manager with systemd

bootctl –path=/boot install

nano -w /boot/loader/loader.conf

Place this content:

default arch
timeout 3
editor 0


nano -w /boot/loader/entries/arch.conf

title ArchLinux
linux /vmlinuz-linux
initrd /initramfs-linux.img
options cryptdevice=UUID=2a4ee265-b0a6-466d-9c94-a05f167236ad:volume root=/dev/mapper/volume-root quiet rw

You can get your luks disk Id by running the command:

blkid

Everything is done, now it’s time to start the computer from the new installation.

exit

umount -R /mnt

sync

systemctl reboot

UFW correctly setting

Uncomplicated firewall (ufw) is commonly the simplest choice in case you need a firewall through software (written in python). El software will apply the parameters of configuration in the iptables of our system to speed up the process of configuration of new rules. Iptables rules apply directly at the kernel level, so check if you have added Netfilter to your kernel.

Debian || Ubuntu:

apt install ufw

ArchLinux:

pacman -S ufw

Gentoo:

emerge -a ufw

If you are using the default kernel configuration of your distribution, already have all the modules and drivers installed then you doesn’t need to worry about. If not, check if your loaded kernel configuration has the Netfilter option is enabled.

zcat /proc/config.gz | grep CONFIG_NETFILTER=

CONFIG_NETFILTER=y

In case there is an “n” recompile your kernel with the added Netfilter functionality.

After installing ufw, we can start using it inmediatly. First of all, you are probably connected to your server using an ssh connection so check the port in wich you have your remote session established with. We need to check that for allowing the connections on that port before enabling ufw on our system. In my case it is:

ufw allow 22/tcp

Previously we have said that ufw is based on iptables, so that these changes that we have made through ufw, have had an immediate effect that we can see in more detail.

iptables -nL

It is time for the rules that we have set to enter into action

ufw enable

We can check all the table information from the previous iptables command in a summary

ufw status

By default ufw is gonna act as a DROP (aka DENY) with all the unwanted traffic, wich results in a “timeout” result in case of sending a request wich is not allowed on our firewall. If you prefer to obtain a response for example if the port is closed, you can chenge that behaviour from /etc/default/ufw and change DEFAULT_INPUT_POLICY=”REJECT”