Mounting various image files in Linux

Hyper-v disk files, with extensions such as vhdx and vhd can be mounted under linux directly with the libguestfs-tools tool, mind you, many packages will be installed… mostly libraries relating to the formats it is capable of mounting

sudo apt-get install libguestfs-tools

Now, the following command should mount the disk inside the virtual machine

guestmount --add diskname.vhdx --inspector /mount/dir/name

If the system does not detect a system, you will need to tell guestmount what partition to mount, to explore the partitions, you will need ….

 virt-filesystems -a disk.vhdx

Once you know your partitions

guestmount --add disk_d.vhdx -m /dev/sda1  /hds/loop

If you would rather mount the disk as read only, you can add the –ro switch

guestmount --add yourVirtualDisk.vhdx --inspector --ro /mnt/anydirectory

To unmount, you will need to use the following command

guestunmount /mnt/anydirectory

Sunshine and moonlight

VNC and RDP are great and all, and for so many purposes, they are the goto solution for remoting into a machine.

Now, another solution which is great (And much better if you have the bandwidth) is to broadcast your screen video and do all the work on the server rather than the client

The solution used to be nvidia’s game stream, which was abandoned by nvidia, the new solution based on nvidia would be the sunshine (Server) and moonlight client

The sunshine+moonlight duo work on almost every platform I need, Windows, Mac, Android, iOS, Even LG TVs running web OS… in short, it is a more universal solution. You can even create a virtual non existent monitor under linux and stream that to a different device !

So, let us start with the server (Sunshine)

Installing sunshine on debian is very easy as a .deb installation file is provided, sunshine is not yet in the debian repositories, but if i understand the license correctly, it can be some time in the future

Now, go to the sunshine website, and download the deb file., in my case, I visit this webpage, and download the sunshine-debian-bookworm-amd64.deb file

Now, from the command prompt, su (to run as root), then cd to the directory where your deb file resides, then “sudo apt install ./sunshine-debian-bookworm-amd64.deb”, We should now have the server running and waiting to be opened in the web browser, Now, on the command line , type “sunshine”

Point a web browser to https://localhost:47990/, ignore the problem with self signed certificates, and set your username and password

Now, your debian computer is running a sunshine server, go to any other machine where you want to install the client (moonlight) from here , and connect to your server by its IP address.

You are done !

GPU PCIe passthrough on KVM

Before you start

This may look like a long post at first, but in reality, it is but a few commands, the rest is output and small simple explanations, so don’t be discouraged by the length, it is really neither complicated nor lengthy.

Yet, you do need to check the hardware requirements before you get your hands dirty, you will find them in the “Minimum hardware requirements” section of this post

Continue reading “GPU PCIe passthrough on KVM”

Mounting QCOW2 (KVM/QEMU) directly

First, the tools you need

apt-get install qemu-utils

Now, enable NBD

modprobe nbd max_part=8

Once that is enabled, connect the file as a block device

qemu-nbd --connect=/dev/nbd0 /hds/usb/virts/Windows/main.qcow2

Now, the block device should appear like any other, alongside the partitions inside !

fdisk -l

On my machine, this resulted in

Disk /dev/nbd0: 95 GiB, 102005473280 bytes, 199229440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc5324c42

Device      Boot     Start       End   Sectors  Size Id Type
/dev/nbd0p1 *         2048    104447    102400   50M  7 HPFS/NTFS/exFAT
/dev/nbd0p2         104448 198138958 198034511 94.4G  7 HPFS/NTFS/exFAT
/dev/nbd0p3      198139904 199225343   1085440  530M 27 Hidden NTFS WinRE

This disk was around 40GB, but fdisk will see the number corresponding to the largest allowed size, 100GB in this case ! let us mount the drive

mount /dev/nbd0p2 /hds/loop

Now, in this case in particular, like any other block device that held the windows operating system, more often than not, you will get the message saying

The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
Falling back to read-only mount because the NTFS partition is in an
unsafe state. Please resume and shutdown Windows fully (no hibernation
or fast restarting.)
Could not mount read-write, trying read-only

The solution to that is simple, follow the following two steps to remedy the issue and then force mount the file by using remove_hiberfile

ntfsfix /dev/nbd0p2
mount -t ntfs-3g -o remove_hiberfile /dev/nbd0p2 /hds/loop

The result of NTFSFIX was

Mounting volume... The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
FAILED
Attempting to correct errors...
Processing $MFT and $MFTMirr...
Reading $MFT... OK
Reading $MFTMirr... OK
Comparing $MFTMirr to $MFT... OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition... OK
Going to empty the journal ($LogFile)... OK
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/nbd0p2 was processed successfully.

And the following mount command worked as you would expect, silently

Step by step Unprivileged containers on Debian Bookworm

The full version of this, with an explanation of everything is here, this one is written for copy-paste and speed.

This version is meant to create unprivileged LXC containers owned by root subordinates, which in my opinion provides the best balance of security and flexibility.

  • Install Debian 12 (bookworm) on a computer or virtual machine or what have you.
  • I personally enable root access under SSH, so all the commands you see here are run as root, you may use another user with sudo if you wish, but i execute as root
  • Execute the following to install LXC (I am installing LXC and KVM) but you might want to remove KVM
apt-get update

apt-get install bridge-utils lxc libvirt-clients libvirt-daemon-system debootstrap qemu-kvm bridge-utils virtinst nmap resolvconf iotop net-tools

Most installations will have 2 users, root and another username you chose while installing the operating system,

Unprivileged containers made simple on Debian 12 (Bookworm)

IMPORTANT NOTE: This is the full version, if you just want to come in, copy some commands, and end up making unprivileged containers under root, THERE IS A SEPARATE POST FOR THAT HERE.

0- Intro

Don’t let the length fool you, I am trying to make this the simplest and fastest yet most comprehensive tutorial to having LXC (both privileged and unprivileged) up and running on debian bookworm !

I sent a previous version of this to a friend to spare myself the need to explain to him what to do, and he found the tutorial confusing ! instead of the old arrangement, having colors to denote what lines are for what task, I have decided to SEPARATE THIS INTO PARTS….

  1. Intro – About this post (You are already in it)
  2. LXC info
  3. Shared system setup (Privileged and unprivileged)
  4. Privilaged LXC step by step
  5. Shared setup for unprivileged containers
  6. Unprivileged LXC run by new user, step by step
  7. Unprivileged LXC run by root user, step by step

I hope this clears things up, the color codes will still exist, mostly because I have already done the work !

Why yet another tutorial ?

Most of the tutorials online focus on creating an extra user to use with LXC, that is one way to do it with a few drawbacks, the other way is to create a range of subordinate IDs for the root user, the advantages of this way of doing it are related to “Autostart” and filesystem sharing between host and guest.

As per usual, the primary goal of every post on this blog is my own reference, the internet is full of misleading and inaccurate stuff, and when i come back to a similar situation, I don’t want to do the research all over again.

Continue reading “Unprivileged containers made simple on Debian 12 (Bookworm)”

Tunneling Firefox traffic through SSH – Putty

I will here assume you already have a remote Linux machine that you can SSH into with putty, the instructions are simple from this point on

Putty Setup

1- Basic putty settings, assuming you have already downloaded putty from chiark.greenend.org.uk, now open putty, enter the IP of the server you wish to tunnel through, and save it with a name, the steps are…
– Open putty,
– enter the IP of your remote machine
– give it a name of your choice
– save (You don’t need to save now, you will save again in a bit, but you can do it anyway)

2- Go to Connection and expand it, then expand SSH, then select Tunnels, this will show a dialogue such as the below, fill in the data as follows

  • A Source port between 1025-65536 (of your choice), i chose 8081 but you can chose any other in that range
  • Check Dynamic and Auto, the click Add

3- From the menu on the left, go back to Session, and click the save button again (So that the new tunnel settings are saved for next time)

4- You are almost done, Now double click the saved session name or select it and hit open, the remote machine should now prompt you to enter a username and a password, once you enter those, you have a tunnel ready on your localhost (127.0.0.1) on port 8081, next we will setup Firefox to use that tunnel

Firefox setup

1- Go to firefox settings (Click the accordion menu to the right, and chose settings), once open, scroll down under general, until you find the Network Settings section, click the settings button in that section

Clicking settings above will show the following popup dialogue, setup your system as follows

  • Manual Proxy Configuration
  • SOCKS Host enter 127.0.0.1 and in the port area of that the port we chose in putty (In my case, 8081)
  • Optional – Add the IP address ranges of the IPs that you do not want to have tunneled through the remote machine
  • For more privacy, and sometimes functionality (When access is blocked from abroad), make sure you tunnel your DNS queries as well (See checkbox below)

Now, to verify that you are conencted to the remote machine, google the following

what is my ip

and google should tell you what your IP address is, at this stage, it should be the same as the remote machine’s IP (Not yours)

Mounting unclean NTFS windows drive in Linux

Whenever i get the following message

mount /dev/sdd1 /hds/sgt2tb
The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
Falling back to read-only mount because the NTFS partition is in an
unsafe state. Please resume and shutdown Windows fully (no hibernation
or fast restarting.)
Could not mount read-write, trying read-only

The command

ntfsfix /dev/sdd1

resolves the issue, and produces the following message

Mounting volume... The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
FAILED
Attempting to correct errors...
Processing $MFT and $MFTMirr...
Reading $MFT... OK
Reading $MFTMirr... OK
Comparing $MFTMirr to $MFT... OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition... OK
Going to empty the journal ($LogFile)... OK
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/sdd1 was processed successfully

The same mount command you see here will now work flawlessly

mount /dev/sdd1 /hds/sgt2tb

I am still unsure what process from the mentioned above is responsible, as this oftentimes pops up on drives that were never system drives, so there is no hibernation file problem

Mounting a remote Linux file system as a Windows drive

You can do this in many ways, the most popular of which is SAMBA, but this is not the software we are using, here we are using SSHFS

The software this post is about is SSHFS, if you are reading this, you probably know what SSH is (Secure shell), and FS stands for File System

Ironically, you will only need to have SFTP and not SSH with shell access, so here is the first surprise, Now, to continue with this tutorial, you might want to visit the page I have posted here to create that user and give him/her access to the directory to be mounted, don’t worry, there is a link back here at the bottom of that page !

So, now that you have created that user account on the remote system, let’s get down to business

You will need 2 peices of software, or 3 if you would like to use private/public key authentication

For the following software, look on their websites for the latest installers for your version of Windows (Usually you are looking for the msi of the 64bit version of windows)

1- WinFsp, short for Windows File System Proxy, What this basically does is enabled the developer of SSHFS-Win to make it look like a windows drive, not some separate SFTP application where you have to move the files manually, when you present it as a drive, you can modify files directly on it, which is the main advantage, and it will do the work in the background, it is a driver that presents itself on/to windows as a disk, while cheating the disk contents from another application, the github page for it is at https://github.com/winfsp/winfsp, or to save you time, Just go directly to the download page here https://github.com/winfsp/winfsp/releases/tag/v1.11 , When presented with optional components, if you are not a developer, you will only ever need the Core package, which is the installer’s default

Once WinFsp is installed, we are done with the part that allows us to display file systems that are not really filesystems, the next step is to have something feed that with data from an actual filesystem somewhere else ! via SFTP, and that software would be

2- SSHFS-Win, which is the system that sits in the middle, between the SFTP server, and WinFsp which is an illusion of a hard drive on your windows machine ! it’s home on github is at https://github.com/winfsp/sshfs-win, To get the latest from this one, go here https://github.com/winfsp/sshfs-win/releases and look for the one that says latest (Not pre-release), download and install it

There is no software to install on the remote side, as most Linux systems already have the functionality ! and you have already setup a user in the previous post that I pointed you to a minute ago, So let us mount !

Now, you can (But don’t do it just yet) open file explorer in Windows, right click “This PC”, and click on Map Network Drive, A dialogue appears, enter your connection string, which should be something like

\\sshfs\username@serverhostname\

You should then be prompted with a password dialogue box, you enter the SFTP password, and you should now be all set, but why are we not doing this right now ? we are not doing this because when you create files in that drive, they will remotely have rwx permissions for owner, and no permissions for group or others, wo work around this, you need to pass the following arguments to the mount

webdev@10.10.20.41:/

create_file_umask=0000,create_dir_umask=0000,umask=0000,idmap=user,StrictHostKeyChecking=no

which is only available via command line and does not survive reboots, a better alternative is to use sshfs-win-manager, which seamlessly mounts those remote file systems using SFTP , the long and short of it is that it just works


Another program that has a different set of permission issues (I can write files, but can’t write to them again even though i own the files on the remote system and the permissions should allow) is SiriKali (https://github.com/mhogomchungu/sirikali), you should be able to find the line to download for your platform here (https://mhogomchungu.github.io/sirikali/)

SiriKali also allows you to use other types of authentication which are beyond the scope of this post

So in SiriKali, you need to fill the above information, luckily that information is loaded by default.

Remember to select the checkboxes you need,

Static IP on Hyper-v (Debian Guests)

One problem i face when developing using Hyper-v is that I need static addresses, and the default switch keeps changing the ip range

The simplest solution to this is to create a new switch of type internal ! this only connects the virtual machines to each other (Static IP etc…), and can not access the internet

Right after creating an INTERNAL switch in the switch manager, you go to “Manage network adapter settings” on the host computer, and assign an IP such as 10.10.20.1 to the adapter and a subnet of 255.255.255.0, no gateway, and nothing but those IPs.

Once that is done, you add a second adapter to all the virtual machines, and in the /etc/network/interfaces file, you leave eth0 the way it was (For internet) and add a metric 10 under the last line for eth0, then add the following stanza for the new adapter (Assuming eth1), eth 1 has a higher cost in it’s metric, so unless the remote is on the eth1 subnet, it will go through the eth0

auto eth1
iface eth1 inet static
 address 10.10.20.41/24
 metric 100

And you are done, those virtual machines can address each other with their 10.10.20.x addresses, and access the outside world via the other network interface.