Tag Archives: software libre

Pfsense + UDM + VLANs: The perfect home network

A couple weeks ago I did a mayor reconfiguration on my home network, I migrated from a single flat insecure network in where any device was able to talk to any other to a more secure design in where the network is segmented (IoT devices, guests, home lab, etc) and where I control who has access to what resources via firewall rules and other tools.

My original home network consisted of a single Google Wifi router, if you are interested the device it’s limited but will get the job done. However I wanted to learn more about networking and in particular how to configure a couple of monitoring tools, network packet inspection, security, firewall rules, etc. So I started looking at networking appliances that will let me do more advanced configurations and I quickly found about Pfsense (Protectli Vault) so I got one.

Additionally, as a birthday gift from @perrohunter, I got The Dream Machine from Ubiquiti (usually you will use one or the other) so I had two routers now. 

I had to integrate them together but I faced a couple of issues during the process to the point where I got locked out from the network and I had to reset the devices multiple times, either the Pfsense or the UDM would work but not both of them at the same time but after some time it’s finally working so I decided to document the process in case it helps someone in the future.

Designing the network

The main goal was to have a clear separation between IoT devices, guest devices and my home devices so i came out with this design.

Disclaimer: I’m a security software engineer but I know a thing or two about networking, if you see something wrong or do you think this design can be improved in any way please let me know.

As you can see, I’m putting the Pfsense at the edge of the network so I have full control over the traffic. I’m using the UDM as an access point only because most of the routing and DNS resolution will be done by Pfsense. The home network consists of 3 VLANs.

IoT network VLAN 30

All my smart lights, roomba, smart locks, cameras will be here, these devices cannot communicate to the other networks or connect to the Internet. Only wireless devices will connect to this network.

Guests network VLAN 50

Occasionally I get visitors at my place, guests can connect to this network and enjoy access to the Internet however devices here will not be able to talk to devices on the IoT nor the LAN network. TODO: I want to put some rules in place so guests’ devices are fully isolated from each other. Only wireless devices will connect to this network.

LAN 

This is the main network and it’s a combination between wired and wireless devices, my work stations, laptops, mobile devices, home servers, smart tv, gaming consoles, etc. These are devices that I trust and most of them have static IPs and dns names.

Setup

I’m not going to explain in detail how to do the initial configuration for the Pfsense or the UDM, there are thousands of videos and tutorials that can guide you through that, instead I’ll focus on the parts I struggled the most and the “hacks” I applied to make this work.

Pfsense setup

These devices will usually come with two ports, WAN and LAN. I had to connect the Ethernet cable from the modem to the WAN port (also called an interface) and that will be enough for the device to talk to the internet in most cases. After that, during the initial configuration Pfsense asked me to configure the LAN interface, there I chose the network IP, IP range, etc In my case I selected 10.13.37.1/24 as my network IP range.

You can tweak and do some more advanced configurations under Services > DHCP Server > LAN

DHCP got configured automatically for this interface so I didn’t worry about it.

After that I grabbed another Ethernet cable and connected it into the Pfsense LAN port and the UDM WAN port.

The Dream Machine (UDM) setup

Here is where the issues begin, I connected the Ethernet cable to the UDM, the app guided my through the initial configuration, then I created the initial Wireless network and everything seemed to work fine however after looking at Status > DHCP Leases on my Pfsense I could not see any of my wireless devices, that was weird.

I logged in into the Dream Router management console and I could see my wireless network, the default network and the wan interface. I also could see all my connected devices, however the assigned IP addresses were in the 192.168.1.1/24 range not the 10.13.37.1/24. So I had some idea about what was happening, UDM had its own DHCP server and was assigning the IP addresses itself.

I start trying many different things, some of them were:

  • Disabling DHCP in the default network of the UDM didn’t work.
  • Changing the network range in the default network of the UDM to 10.13.37.1/24 didn’t work, UDM was complaining that the range conflicts with the IP assigned to it (10.13.37.2).
  • Created an additional network on the range I wanted 10.13.37.1/24 didn’t work, devices from here were not able to see the Pfsense.

I tried many more things and after a couple weekends of trial and error I found the winning combination of steps, this is probably the most important part of this article.

  • Disconnect the Ethernet cable from the UDM WAN port, this cause the UDM to lose the IP assigned by the Pfsense
  • Change the default network configuration in the UDM to use the 10.13.37.3/24 network, this network will overlap with the 10.13.37.1/24 network in Pfsense but it’s ok, also set DHCP Mode to none.
  • In the UDM go to Internet > default WAN and select manual configuration, here I’m setting the primary DNS server as 10.13.37.1 (Pfsense) and IPv4 configuration has to be as follow

Here I’m telling UDM the next hop will be at 10.13.37.1 (Pfsense), also I want the UDM to use the static IP 10.13.37.2, and the subnet mask will be 255.255.255.248 which ended being the “hack” that allow me to use the 10.13.37.x range on the default network

Finally plug the Ethernet cable again into the UDM but this time into any of the LAN ports not the WAN (the little world icon), avoid the WAN port seriously!.

The reason why I want the default network in the UDM to be an overlap of the 10.13.37.1/24 network in the Pfsense was because otherwise I would lose access to the UDM management console, I’m still trying to figure out why is that but my guess is even if the UDM is accessible from the Pfsense network on 10.13.37.2 IP address when I try to go to there (if the default network range is configured to be 192.168.1.1 on UDM) it won’t let me in because of some validation on UDM, to avoid this I ended creating a dedicated wireless network just to recover access (after getting locked out multiple times).

Using the above configuration my devices in the 10.13.37.1/24 range are able to talk to Pfsense (10.13.37.1) and also the UDM (10.13.37.3) and finally I’m able to see and control my devices from the Pfsense as well.

VLANs

Network interfaces

The main network is working fine now what? I started creating additional VLANs and firewall rules for the guests and the IoT networks. On the Pfsense I went to  Interfaces > Assignments > VLANs and added the two VLANs. It’s very important to select LAN as the parent interface because all the traffic is going to come from that port.

For no particular reason I chose tag 30 for the IoT VLAN and tag 50 for the guest VLAN, don’t forget to assign the new VLANs to the LAN interface and create the new networks.

To be consistent I decided the guests network range will be 10.13.50.1/24 and the IoT will follow 10.13.30.1/24

DHCP Server

Now it was time to configure the DHCP server for the new networks, I went to Services > DHCP Server and made sure the enable DHCP box was checked, additionally I configured the assignable IP range. I did this for both networks.

Firewall rules

According to my original design the guests and IoT network have to be isolated from everything else and in particular the IoT devices should not have any access to the Internet, let’s do that very quickly by configuring firewall rules on Pfsense (Firewall > Rules).

These are the rules applied to the IoT_VLAN, here I’m telling Pfsense to block any incoming connection from the IoT network to the home or guests network, I’m also blocking the access to the Pfsense management console itself on port 8443 and 3000. This firewall by default will block any egress traffic in the network and because I’m not saying otherwise this network will not have access to the Internet.

The guest firewall rules are pretty much the same with the exception that I will allow users to access the Internet (see the last rule).

The Dream Machine

At this point I was done with the Pfsense part but I was missing one last import piece, configuring the access method for the IoT and guests devices so for that I had to return to the UDM management console and create a couple of wireless and network configurations.

Guest Network

I created the new guest network configuration, most of the default values were ok but I had to pay special attention to the VLAN ID section, this one has to match to the one I configured on Pfsense (tag 50). Also is very important to set DHCP Mode to None

I created the Wifi network and told UDM to use the guest network, all packets will be marked (tag 50) and managed by the guest VLAN.

IoT Network

I repeated the previous steps but this time for the IoT network, I proceeded to create the network, added the right VLAN (tag 30) and disabled DHCP, then configured the wifi network as well.

Testing

Once everything is configured the way I wanted I tested by connecting a couple of devices to the IoT network and monitored the traffic with the help of ntopng (maybe I will write a blogpost about it in the future), there I confirmed there was not a single request to a remote address.

Conclusion

Designing a network is one of the most fun things you can do in IT. The main reason for me to get the Pfsense was because I wanted to learn more about networking and have hands-on experience with several networking and security tools. VLANs, Firewall rules, DHCP, DNS, packet inspection etc are good skills for a security engineer but these are only the tip of the iceberg for a network engineer.

Build your own private cloud at home with a Raspberry Pi + Minio

Early this year I got one of those widescreen 5k monitors so I could work from home, the display is so cool but the sad thing is it only comes with 2 USB ports. I have a wired mouse and keyboard so when I wanted to connect an external hard drive for copying and backing up files it was always a pain in the neck.

I remembered I have an old Raspberry PI2 I brought with me from México so last weekend I decided to work on a small personal project for solving this issue once and for all, I finished it and it’s working very well so I thought on writing a blogpost about it so more people can build its own private cloud at home too.

Install Raspbian

The first thing was to install a fresh version of raspbian into the raspberry pi, I got it from https://www.raspberrypi.org/downloads/raspbian/, I wanted something minimal so I got the Raspbian Buster Lite image, this version of raspbian doesn’t come with a graphical interface but it’s fine because ssh it’s all what we need.

Insert the SD card into your machine, I’m using a macbook pro so I have to use an adapter, once the card is there you can verify using the df command, tip: you can easily identify your SD card by the size reported by df -h.

[bash]
df -h

Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1s5 466Gi 10Gi 246Gi 5% 487549 4881965331 0% /
devfs 338Ki 338Ki 0Bi 100% 1170 0 100% /dev

..
/dev/disk2s1 <————- my SD card
[/bash]

Before copying the image first you need to unmount the device using sudo umount /dev/disk2s1 after that you can use the dd command.

[bash]
sudo dd bs=1m if=./2020-02-13-raspbian-buster-lite.img of=/dev/disk2s1
[/bash]

Optionally you can do all this process in a more friendly way by installing Raspberry Pi imager tool https://www.raspberrypi.org/downloads/, you need to insert your sd card, choose the os, choose the sd card and the click the write button.

Once you have your fresh version of Raspbian installed it’s time to verify the Raspberry is working, the easiest way to do that is to connect a monitor and keyboard to it, so I did it.

When you connect the raspberry to the power the green led should start flashing, if that doesn’t happen is probably a sign of a corrupted EEPROM and you should look at the Recovery section of https://www.raspberrypi.org/downloads/.

Access the Raspberry Pi remotely

Alright, if you get to this point means your raspberry is fine, next step is to connect it to your network, I connected mine to my switch using an ethernet cable, before ssh into the raspberry first we need to get its IP, there are multiple ways to get the IP address assigned to your raspberry, I used nmap https://nmap.org/ to quickly scan my local network for new devices.

[bash]
nmap -sP 192.168.86.0/24

Starting Nmap 7.80 ( https://nmap.org ) at 2020-03-29 19:55 PDT
Nmap scan report for testwifi.here (192.168.86.1)

..
Nmap scan report for raspberrypi (192.168.86.84)
Host is up (0.0082s latency).

..
Nmap done: 256 IP addresses (10 hosts up) scanned in 2.55 seconds
[/bash]

Ok from now on I’m going to start referring to the raspberry as nstorage (network storage), on my local machine I added a new entry to /etc/hosts with this information.

[bash]
# Minio running in raspberry pi in home network
192.168.86.84 nstorage
192.168.86.84 raspberry
[/bash]

I also added a new entry on ~/.ssh/config so it is easier to connect via ssh.

[bash]
Host nstorage
User pi
Hostname nstorage
Port 22
ServerAliveInterval 120
ServerAliveCountMax 30
[/bash]

You can type on your terminal ssh nstorage, and login using the default credentials: pi / raspberry.

[bash]
ssh nstorage

Linux raspberrypi 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Mon Mar 30 03:27:49 2020 from 192.168.86.64
[email protected]:~ $
[/bash]

First thing you should do is change the default password using the passwd command http://man7.org/linux/man-pages/man1/passwd.1.html.

One thing I always like to do is to add the public ssh key of my machine (my macbook pro) to the list of authorized_keys on the remote server (nstorage), you can do this by copying your public key: cat ~/.ssh/id_rsa.pub | pbcopy and then in nstorage in the /home/pi/.ssh/authorized_keys (create the file if it doesn’t exist) file append the key to the end.

[bash]
[email protected]:~/.ssh $ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvxqCsC2RWVfWfix/KT1R8eZ9zN5SXoZ8xV8eCsk47AZUkZKBdCLxp0arhS2/+WpjRAFuR4+XgmnWlu/rQYzWGaqv/sm5420zaF6fpOaeFXEuLGVP7Nb4e1oPR1tNbzZ7OLJs1FVZIk8rBeTfLh2+UMU8Lut+rKtd9FbW4LdTimscg8ufeFZ1bKWTPih4+o3kYEdSFpMz0ntKDqKA7g3Kvq6PbhUxcICA/KrJbjxTjuOelfqsfTz7xrJW/sII5QETTqL93ny7DlPdVdM2Qw6C/1NZ1hV7ZgpihFlD+XKhdqdugG9DgjzgKvdNx63idswCRJKmdxHZN+oM33+bASHMT [email protected]
[/bash]

That way next time you ssh into nstorage (the raspberry) the login process will be automatic.

Install Minio

You are on a fresh raspbian system, first thing you should do is update the existing software.

[bash]
sudo apt-get update
sudo apt-get upgrade
[/bash]

After that lets download the minio server and the minio client, we also create symbolic links for both binaries.

[bash]
wget https://dl.minio.io/server/minio/release/linux-arm/minio
wget https://dl.minio.io/client/mc/release/linux-arm/mc
sudo ln -s /home/pi/minio /usr/bin/minio
sudo ln -s /home/pi/mc /usr/bin/mc
[/bash]

At this point you can start a simple minio server with:

[bash]
[email protected]:~ $ mkdir ~/data
[email protected]:~ $ minio server ~/data
Endpoint: https://192.168.86.84:9000 https://127.0.0.1:9000
AccessKey: minioadmin
SecretKey: minioadmin

Browser Access:
https://192.168.86.84:9000 https://127.0.0.1:9000

Command-line Access: https://docs.min.io/docs/minio-client-quickstart-guide
$ mc config host add myminio https://192.168.86.84:9000 minioadmin minioadmin

Object API (Amazon S3 compatible):
Go: https://docs.min.io/docs/golang-client-quickstart-guide
Java: https://docs.min.io/docs/java-client-quickstart-guide
Python: https://docs.min.io/docs/python-client-quickstart-guide
JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
.NET: https://docs.min.io/docs/dotnet-client-quickstart-guide

Detected default credentials ‘minioadmin:minioadmin’, please change the credentials immediately using ‘MINIO_ACCESS_KEY’ and ‘MINIO_SECRET_KEY’
[/bash]

In your local machine go to http://nstorage:9000/minio and you will see the following screen.

We are almost there, you have a minio server running in your raspberry pi, you can start uploading files and creating buckets if you want, but first let’s add some security.

Securing your Minio

Right now all the traffic between you and nstorage (your minio server) is unencrypted, let’s fix that quickly, I used mkcert https://github.com/FiloSottile/mkcert by Filippo Valsorda for quickly generate certificates signed by a custom certificate authority, sounds scary but is actually quite simple.

In the raspberry we are going to create the following folders to hold the certificates.

[bash]
mkdir ~/.minio/certs/CAs
mkdir ~/.mc/certs/CAs
[/bash]

In your local machine we generate and push the certificates to the raspberry, dont forget to also push the public key of your local certificate authority created by mkert under /Users/$USER/Library/Application Support/mkcert/rootCA.pem.

[bash]
$ mkcert nstorage
Using the local CA at "/Users/alevsk/Library/Application Support/mkcert" ✨

Created a new certificate valid for the following names 📜
– "nstorage"

The certificate is at "./nstorage.pem" and the key at "./nstorage-key.pem" ✅

$ ls nstorage*
nstorage-key.pem nstorage.pem
$ scp ./nstorage* [email protected]:~/.minio/certs
$ scp ./rootCA.pem [email protected]:~/.minio/certs/CAs
$ scp ./rootCA.pem [email protected]:~/.mc/certs/CAs
[/bash]

That’s it, you have now a secure connection with your Minio, if you go to your browser you can HTTPS this time.

Nstorage certificate is valid and trusted by your system because was generated by your local certificate authority, every device that wants to access this server need to trust the CA as well, otherwise it will get a trust error.

Mount external drive

Alright, so far you have a secure Minio running on the raspberry pi, in my case I used a 16GB SD card, which was not enough for storing all my data and the whole point was to access my external drive files remotely, so let’s do that now. But first instead of start Minio manually let’s create a bash script and change the default credentials.

Create a new file using vim or your editor of choice: vim start.sh

[bash]
#!/bin/bash

export MINIO_ACCESS_KEY=SuperSecretAccessKey
export MINIO_SECRET_KEY=SuperSecretSecretKey
export MINIO_DOMAIN=nstorage
export MINIO_DISK_USAGE_CRAWL=off

minio server ~/data
[/bash]

Save the above lines and then give execution permissions to the script: chmod +x start.sh
Now you can start your Minio running ./start.sh

[bash]
[email protected]:~ $ ./start.sh
Endpoint: https://192.168.86.84:9000 https://127.0.0.1:9000
AccessKey: SuperSecretAccessKey
SecretKey: SuperSecretSecretKey

Browser Access:
https://192.168.86.84:9000 https://127.0.0.1:9000

Command-line Access: https://docs.min.io/docs/minio-client-quickstart-guide
$ mc config host add myminio https://192.168.86.84:9000 SuperSecretAccessKey SuperSecretSecretKey

Object API (Amazon S3 compatible):
Go: https://docs.min.io/docs/golang-client-quickstart-guide
Java: https://docs.min.io/docs/java-client-quickstart-guide
Python: https://docs.min.io/docs/python-client-quickstart-guide
JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
.NET: https://docs.min.io/docs/dotnet-client-quickstart-guide
[/bash]

Now connect your external hard drive to one of the USB ports, I had some issues while doing this, Raspbian was not listing the device under /dev so make sure to increase the USB ports power via configuration in /boot/config.txt, add max_usb_current=1 to the end of the file.

[bash]
[email protected]:~ $ cat /boot/config.txt
# For more options and information see
# http://rpf.io/configtxt
# Some settings may impact device functionality. See link above for details

..
# Increase power available to USB ports
max_usb_current=1
[/bash]

Reboot the raspberry and plug your drive again, if everything went right you should be able to see your external drive using fdisk.

[bash]
$ sudo fdisk -l
Disk /dev/sda: 4.6 TiB, 5000981077504 bytes, 9767541167 sectors
Disk model: Expansion Desk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 24A09C07-313E-43B6-A811-FAF09DAB962C

Device Start End Sectors Size Type
/dev/sda1 34 262177 262144 128M Microsoft reserved
/dev/sda2 264192 9767540735 9767276544 4.6T Microsoft basic data
[/bash]

You can mount the device using the mount command https://linux.die.net/man/8/mount.

[bash]
[email protected]:~ $ sudo mount -t ntfs /dev/sda2 /home/pi/data
[email protected]:~ $ ls -la data
total 9032
drwxrwxrwx 1 root root 8192 Mar 30 08:19 .
drwxr-xr-x 9 pi pi 4096 Mar 30 08:27 ..
drwxrwxrwx 1 root root 65536 Mar 26 22:53 anime
drwxrwxrwx 1 root root 20480 May 5 2019 anime_movies
drwxrwxrwx 1 root root 0 Jan 4 2019 backup
drwxrwxrwx 1 root root 4096 Jan 4 2019 books
drwxrwxrwx 1 root root 4096 Jan 4 2019 dev
drwxrwxrwx 1 root root 16384 Feb 12 2017 documents
drwxrwxrwx 1 root root 0 Feb 6 2017 download
drwxrwxrwx 1 root root 12288 Feb 12 2017 games
drwxrwxrwx 1 root root 4096 Jan 4 2019 images
drwxrwxrwx 1 root root 4096 Feb 10 2017 manga
drwxrwxrwx 1 root root 4096 Mar 29 07:48 .minio.sys
drwxrwxrwx 1 root root 65536 Mar 30 01:41 movies
drwxrwxrwx 1 root root 0 Jan 4 2019 music
drwxrwxrwx 1 root root 0 Feb 6 2017 pentest
drwxrwxrwx 1 root root 12288 Jun 2 2019 series
drwxrwxrwx 1 root root 4096 Jun 2 2019 software
drwxrwxrwx 1 root root 0 Jan 25 20:51 .Trashes
drwxrwxrwx 1 root root 0 Jun 21 2019 videos
[email protected]:~ $
[/bash]

Restart your minio server and this time when you go to the browser you will see all your files there.

You can list all the files and buckets using the minio client (mc) from your local machine or using the mc binary inside the nstorage raspberry.

[bash]
$ mc config host add nstorage https://nstorage:9000 SuperSecretAccessKey SuperSecretSecretKey
$ mc ls nstorage

[2020-03-26 15:53:09 PDT] 0B anime/
[2019-05-04 18:25:59 PDT] 0B anime_movies/
[2019-01-03 23:00:08 PST] 0B backup/
[2019-01-03 23:04:29 PST] 0B books/
[2019-01-03 23:48:04 PST] 0B dev/
[2017-02-11 17:09:28 PST] 0B documents/
[2017-02-05 16:45:21 PST] 0B download/
[2017-02-11 16:03:31 PST] 0B games/
[2019-01-03 23:06:48 PST] 0B images/
[2017-02-10 11:50:31 PST] 0B manga/
[2020-03-29 17:41:41 PDT] 0B movies/
[2019-01-03 22:48:15 PST] 0B music/
[2017-02-05 22:14:30 PST] 0B pentest/
[2019-06-02 14:33:34 PDT] 0B series/
[2019-06-01 21:29:46 PDT] 0B software/
[2019-06-20 20:20:56 PDT] 0B videos/
[/bash]

You can download every file you want, upload files and also stream media. Go to your Minio browser and select any video you like, click on the “3 dots” icon on the right and click the share icon.

Minio will generate a pre-signed URL that you can use on VLC, click on File > Open Network and paste the video URL.

Click the open button and enjoy your videos.

Everything is great so far, you are able to access all your files from any device in your network but if your raspberry loses power and reboot you will need to mount the external drive and start the Minio server manually again so let’s automate that.

Mount the external drive with fstab

On linux by default every drive listed in /etc/fstab will be mounted on startup, there are many ways to mount drives but the recommended way is using UUID or PARTUUID instead of the name.

[bash]
[email protected]:~ $ sudo blkid



/dev/sda2: LABEL="Arael" UUID="62F048D0F048AC5B" TYPE="ntfs" PTTYPE="atari" PARTLABEL="Basic data partition" PARTUUID="5206da84-ded1-43b6-abf2-14b5950c4d7c"
[/bash]

Locate the PARTUUID of your own drive, mine was 5206da84-ded1-43b6-abf2-14b5950c4d7c, and then add it at the end of your /etc/fstab file.

[bash]
$ cat /etc/fstab

proc /proc proc defaults 0 0
PARTUUID=738a4d67-01 /boot vfat defaults 0 2
PARTUUID=738a4d67-02 / ext4 defaults,noatime 0 1
# a swapfile is not a swap partition, no line here
# use dphys-swapfile swap[on|off] for that
PARTUUID=5206da84-ded1-43b6-abf2-14b5950c4d7c /home/pi/data ntfs defaults,errors=remount-ro 0 1
[/bash]

Reboot your raspberry and verify your drive was mounted automatically under /home/pi/data.

Start the Minio server with systemctl

Finally, the last piece of the puzzle is to make minio to start automatically, again, there’s many ways to do this but in this tutorial we will do it with init system or systemctl, let’s create a file called minio.service with the following content.

[bash]
[Unit]

Description=Minio Storage Service

After=network-online.target home-pi-data.mount

[Service]

ExecStart=/home/pi/start.sh

WorkingDirectory=/home/pi

StandardOutput=inherit

StandardError=inherit

Restart=always

User=pi

[Install]

WantedBy=multi-user.target
[/bash]

ExecStart points to the start.sh bash script, After directive will tell the Minio server to wait until the network service is online and the /dev/sda2 drive is mounted by fstab, home-pi-data.mount is a systemd mount unit you can get using the systemctl list-units command.

[bash]
$ systemctl list-units | grep ‘/home/pi/data’ | awk ‘{ print $1 }’
home-pi-data.mount
[/bash]

Copy the file to the /etc/systemd/system directory.

[bash]
cp ./minio.service /etc/systemd/system/minio.service
[/bash]

Start minio as a systemd service using the start command and verify is running with the status command.

[bash]
[email protected]:~ $ sudo systemctl start minio
[email protected]:~ $ sudo systemctl status minio
● minio.service – Minio Storage Service
Loaded: loaded (/etc/systemd/system/minio.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-03-30 10:12:22 BST; 4s ago
Main PID: 1453 (start.sh)
Tasks: 16 (limit: 2200)
Memory: 156.2M
CGroup: /system.slice/minio.service
├─1453 /bin/bash /home/pi/start.sh
└─1456 minio server /home/pi/data

Mar 30 10:12:22 raspberrypi systemd[1]: Started Minio Storage Service.
[/bash]

If everything looks fine, enable the service, Minio will start automatically every time your Raspberry pi boot.

[bash]
sudo systemctl enable minio
[/bash]

Reboot your raspberry pi one last time and verify everything is working as expected, if you are able to see the minio browser at https://nstorage:9000/minio without you having to do anything congratulations you now have your own private cloud at home powered by Minio :).

Happy hacking.

Commands and Code Snippets I usually forget

Some commands and code snippets I use rarely during CTFs or my daily work, but still I need them from time to time and I’m very lazy to remember them. This note may grow over time.

Javascript

Playing with dec, hexa and bin (not really) in JS

[javascript]
String.fromCharCode(0x41) // ‘A’

parseInt(‘0xf’, 16) // 15

var n = 15

n.toString(16) // ‘f’
n.toString(2) // ‘1111’
n.toString() // ’15’

var n = ‘A’
n.charCodeAt() // 65
// dec to hex
n.charCodeAt().toString(16) // ’41’
// dec to bin
n.charCodeAt().toString(2) // ‘1000001’
// dec to hex
parseInt(255).toString(16) // ‘ff’
// dec to bin
parseInt(5).toString(2) // ‘101’
[/javascript]

Simple HTTP GET request using nodejs

[javascript]
const https = require(‘https’);

https.get(‘https://www.alevsk.com’, (resp) => {
let data = ”;
resp.on(‘data’, (chunk) => {
data += chunk;
});
resp.on(‘end’, () => {
//DO something with data
});
}).on("error", (err) => {
console.log("Error: " + err.message);
});
[/javascript]

Simple HTTP POST request using nodejs

[javascript]
const https = require(‘https’)

const data = JSON.stringify({
todo: ‘Buy the milk’
})

const options = {
hostname: ‘whatever.com’,
port: 443,
path: ‘/todos’,
method: ‘POST’,
headers: {
‘Content-Type’: ‘application/json’,
‘Content-Length’: data.length
}
}

const req = https.request(options, res => {
res.on(‘data’, d => {
process.stdout.write(d)
})
})

req.on(‘error’, error => {
console.error(error)
})

req.write(data)

req.end()
[/javascript]

Extract content between regular expression patterns using JS

[javascript]
const message = data.match(/<p>([^<]+)<\/p>/)[1];
const lat = data.match(/name="lat" value="([^<]+)" min=/)[1];
const long = data.match(/name="lon" value="([^<]+)" min=/)[1];
const token = data.match(/name="token" value="([^<]+)"/)[1];
[/javascript]

Linux

Mount NTFS on Linux

[bash]
mount -t ntfs [FILE] [PATH]
mount -t type device directory
[/bash]

Extract extended attributes from NTFS disk

[bash]
getfattr –only-values [FILE] -n [ATTR-NAME] > file
[/bash]

Parsing file with awk and run xargs

[bash]
cat [FILE] | awk ‘{print $1 .. $n}’ | xargs
[/bash]

Python

Start Simple HTTP server with Python

[bash]
python -m SimpleHTTPServer
[/bash]

Inline Python commands

[bash]
python -c ‘print "\x41" * 20’
[/bash]

PHP

Run PHP interactive mode

[bash]
php -a
[/bash]

Security Fest #CTF – Zion write up

Para este reto nos daban un archivo comprimido zion.tar.gz, procedemos a descomprimirlo y obtenemos otro archivo llamado YouKnow.

El archivo no tiene extension pero utilizamos el comando file para ver que tipo de archivo es.

Parece un archivo de Microsoft Word Office y sabemos que los archivos docx en realidad son archivos en formato zip.

Procedemos a descomprimir YouKnow

Obtenemos varios archivos y carpetas, comenzamos a analizarlos de uno por uno, sin embargo no encontramos nada que haga referencia a la bandera del reto. (analice la imagen del conejo con un par de herramientas de esteganografía pero no había nada)

Damos un paso atrás y abrimos el archivo YouKnow en un editor hexadecimal de su elección, you utilice Sublime

Observamos la cabecera estándar PK del formato ZIP

Al ir analizando el archivo, hacia el final, algo salta inmediatamente a la vista.

Parece que hay otro archivo Zip concatenado al primero pero los bytes están en orden inverso (observen como el archivo termina en KP, y vemos algunos strings como lmx que seria xml).

Podemos utilizar python para invertir los bytes del archivo fácilmente.

[python]
open(‘YouKnow_reversed’,’wb’).write(open(‘YouKnow’,’rb’).read()[::-1])
[/python]

Obtenemos el archivo con los bytes invertidos y procedemos a descomprimirlo.

Obtenemos nuevamente varios archivos y carpetas.

Y en donde estaba la imagen anterior del conejo rojo ahora encontramos otra imagen, esta vez de un conejo azul que nos muestra la bandera del reto 🙂

La bandera del reto es sctf{m41nfr4m3_4cc3ss_c0d3_1337_4lw4s}

Bonus

Programe una pequeña herramienta en python llamada reverse bytes para invertir los bytes de un archivo utilizando una cli mas amigable.

[bash]
usage: rbytes.py [-h] [-o OUTFILE] infile

A simple python script for reverse the bytes of a file.

Author: Lenin Alevski Huerta Arias
Year: 2018

positional arguments:
infile Input file

optional arguments:
-h, –help show this help message and exit
-o OUTFILE, –outfile OUTFILE
Output file
[/bash]

Happy hacking 🙂

Docker 101 #2: puertos y volúmenes de un contenedor

docker-image

En el artículo anterior comenzamos con una breve introducción a docker, vimos su instalación, configuración e incluso lanzamos un par de servidores web nginx usando contenedores, en esta ocasión explicare un poco más acerca de los puertos y los volúmenes.

Puertos

Ok, lo primero que explicare será el mapeo de puertos, abrimos una terminal y ejecutamos el siguiente comando:

[bash]
$ sudo docker run –name servidor-web -p 80:80 nginx
[/bash]

El parametro –name sirve para asignarle un nombre al contenedor.

El parámetro -p sirve para realizar el mapeo de puertos y recibe una cadena en el formato PUERTO-HOST:PUERTO-CONTENEDOR, es decir, del lado izquierdo definimos el puerto que nuestro sistema operativo le asignara al contenedor de docker y del lado derecho el puerto en el que realmente se ejecuta el servicio dentro del contenedor, en este caso nginx (suena un poco confuso al inicio así que regresa y léelo de nuevo hasta que lo entiendas)

En el comando anterior estamos mapeando el puerto 80 de nuestra computadora con lo que sea que este corriendo en el puerto 80 del contenedor, es por eso que si vamos a http://localhost veremos el servidor web en ejecución 🙂

nginx

En la consola desde donde ejecutaste el comando podrás ver las peticiones hechas al servidor dentro del contenedor.

docker-cli

Al ejecutar el comando y correr el contenedor abras notado que la consola se queda bloqueada por el servidor web, para evitar eso podemos correr el contenedor en modo detach con el parámetro -d, esto ejecutara el contenedor en segundo plano.

[bash]
$ sudo docker run -d –name servidor-web -p 80:80 nginx
[/bash]

docker_detach

Observa como tan pronto como ejecutamos el comando docker nos devuelve el control de la terminal, cuando ejecutas contenedores de esta forma no olvides que para eliminarlos primero tienes que recuperar su id, el cual puedes obtener haciendo:

[bash]
$ sudo docker ps
[/bash]

y en la primera columna encontraras el ID del contenedor que después deberás de eliminar usando sudo docker rm [CONTAINER-ID], si lo prefieres un tip muy útil para borrar todos los contenedores que hayas creado es ejecutar:

[bash]
$ sudo docker stop $(sudo docker ps -a -q)
$ sudo docker rm $(sudo docker ps -a -q)
[/bash]

El primer comando detiene todos los contenedores que estén en ejecución y el segundo los elimina todos (no puedes eliminar un contenedor que este en ejecución).

Puedes correr todas los contenedores que quieras (o necesites) de nginx en diferentes puertos y con diferentes nombres y cada uno será una instancia completamente diferente del servidor web 🙂
containers

Observa como cada uno de los servidores web corren en un puerto diferente.

multi-docker

Volúmenes

Los volúmenes en docker pueden ser definidos con el parámetro -v y nos ayudan a resolver el problema de la persistencia de datos en los contenedores, un volumen puede ser visto como un mapeo entre un directorio de nuestra computadora y un directorio en el sistema de archivos del contenedor, regresemos a nuestro contenedor de nginx, ¿cómo le hacemos para mostrar un sitio web en nginx en lugar de la página por default?

Lo primero que haremos será crear una carpeta en donde colocaremos el código fuente de nuestro sitio web html (por ahora no trabajaremos con nada dinamico), por ejemplo website

website

Ejecutamos el siguiente comando mapeando el contenido de /home/alevsk/dev/sitio-web hacia /usr/share/nginx/html que es el directorio por default que utiliza nginx para servir contenido a Internet.

[bash]
$ sudo docker run -d –name sitio-web -v /home/alevsk/dev/sitio-web:/usr/share/nginx/html -p 80:80 nginx
[/bash]

La próxima vez que visitemos http://localhost/ veremos nuestro sitio web corriendo.

nginx-web

Puedes replicar este contenedor con el contenido del sitio web tantas veces como quieras, es muy util en un escenario donde necesitas varios ambientes para pruebas, desarrollo, etc.

Eso es todo por ahora, en el siguiente tutorial aprenderemos a crear nuestras propias imágenes de docker (dockerizar aplicaciones), después de eso veremos otra herramienta bastante útil llamada docker-compose para facilitar la orquestación de aplicaciones que utilizan múltiples contenedores.

Saludos y happy hacking.