Category: Linux

You can do this in many ways, the most popular of which is SAMBA, but this is not the software we are using, here we are using SSHFS

The software this post is about is SSHFS, if you are reading this, you probably know what SSH is (Secure shell), and FS stands for File System

Ironically, you will only need to have SFTP and not SSH with shell access, so here is the first surprise, Now, to continue with this tutorial, you might want to visit the page I have posted here to create that user and give him/her access to the directory to be mounted, don’t worry, there is a link back here at the bottom of that page !

So, now that you have created that user account on the remote system, let’s get down to business

You will need 2 peices of software, or 3 if you would like to use private/public key authentication

For the following software, look on their websites for the latest installers for your version of Windows (Usually you are looking for the msi of the 64bit version of windows)

1- WinFsp, short for Windows File System Proxy, What this basically does is enabled the developer of SSHFS-Win to make it look like a windows drive, not some separate SFTP application where you have to move the files manually, when you present it as a drive, you can modify files directly on it, which is the main advantage, and it will do the work in the background, it is a driver that presents itself on/to windows as a disk, while cheating the disk contents from another application, the github page for it is at https://github.com/winfsp/winfsp, or to save you time, Just go directly to the download page here https://github.com/winfsp/winfsp/releases/tag/v1.11 , When presented with optional components, if you are not a developer, you will only ever need the Core package, which is the installer’s default

Once WinFsp is installed, we are done with the part that allows us to display file systems that are not really filesystems, the next step is to have something feed that with data from an actual filesystem somewhere else ! via SFTP, and that software would be

2- SSHFS-Win, which is the system that sits in the middle, between the SFTP server, and WinFsp which is an illusion of a hard drive on your windows machine ! it’s home on github is at https://github.com/winfsp/sshfs-win, To get the latest from this one, go here https://github.com/winfsp/sshfs-win/releases and look for the one that says latest (Not pre-release), download and install it

There is no software to install on the remote side, as most Linux systems already have the functionality ! and you have already setup a user in the previous post that I pointed you to a minute ago, So let us mount !

Now, you can (But don’t do it just yet) open file explorer in Windows, right click “This PC”, and click on Map Network Drive, A dialogue appears, enter your connection string, which should be something like

\\sshfs\username@serverhostname\

You should then be prompted with a password dialogue box, you enter the SFTP password, and you should now be all set, but why are we not doing this right now ? we are not doing this because when you create files in that drive, they will remotely have rwx permissions for owner, and no permissions for group or others, wo work around this, you need to pass the following arguments to the mount

webdev@10.10.20.41:/

create_file_umask=0000,create_dir_umask=0000,umask=0000,idmap=user,StrictHostKeyChecking=no

which is only available via command line and does not survive reboots, a better alternative is to use sshfs-win-manager, which seamlessly mounts those remote file systems using SFTP , the long and short of it is that it just works

---

Another program that has a different set of permission issues (I can write files, but can’t write to them again even though i own the files on the remote system and the permissions should allow) is SiriKali (https://github.com/mhogomchungu/sirikali), you should be able to find the line to download for your platform here (https://mhogomchungu.github.io/sirikali/)

SiriKali also allows you to use other types of authentication which are beyond the scope of this post

So in SiriKali, you need to fill the above information, luckily that information is loaded by default.

Remember to select the checkboxes you need,

One problem i face when developing using Hyper-v is that I need static addresses, and the default switch keeps changing the ip range

The simplest solution to this is to create a new switch of type internal ! this only connects the virtual machines to each other (Static IP etc…), and can not access the internet

Right after creating an INTERNAL switch in the switch manager, you go to “Manage network adapter settings” on the host computer, and assign an IP such as 10.10.20.1 to the adapter and a subnet of 255.255.255.0, no gateway, and nothing but those IPs.

Once that is done, you add a second adapter to all the virtual machines, and in the /etc/network/interfaces file, you leave eth0 the way it was (For internet) and add a metric 10 under the last line for eth0, then add the following stanza for the new adapter (Assuming eth1), eth 1 has a higher cost in it’s metric, so unless the remote is on the eth1 subnet, it will go through the eth0

auto eth1
iface eth1 inet static
 address 10.10.20.41/24
 metric 100

And you are done, those virtual machines can address each other with their 10.10.20.x addresses, and access the outside world via the other network interface.

On linux, when i needed to edit a video I usually go to Kdenlive, I used to think it is THE linux alternative to Adobe Premier Pro, until yesterday, when a friend recommended I try out OpenShot

I am a Gnome user, Kdenlive was designed for KDE, I have always run it in gnome (Unity) and it worked fine, but today, I am feeling adventurous and up for trying OpenShot, openshot too is pyQt which is a bummer, but hey, this is not why we doing this

Reminder
KDE is based on Qt and Gnome is based on GTK. both applications here are meant for KDE, and I don’t really want to install KDE on my machine

This post here is where I will leave my impressions so hang tight.

If you are like me, running only your own projects on a server, you might want to skip authentication

the changes I generally make to the file /etc/redis/redis.conf are

1- At the very beginning, limit the RAM redis can use with the line

maxmemory 2gb

2- Change the supervisor to systemd by modifying the line

supervised no

to

supervised systemd

Now, to test the new config, from the command line, run the following commands

systemctl restart redis-server
redis-cli
ping
config get maxmemory
quit

You have just configured redis and tested your new settings.

The reason I am enabling this in my virtual machine is to develop with android studio under windows or Linux in a dedicated development machine (Let us call it an android development virtual machine), you will need to enable nested virtualization for the virtual android phone that comes with Android studio, there are many occasions where you need nested virtualization, so let us see what we need to do.

1- Check if our system allows nested virtualization with the following line

cat /sys/module/kvm_intel/parameters/nested 

If this returns a Y or a 1, then we are good to go to the next step, if not, then execute the following to enable the feature on the host system

echo 'options kvm_intel nested=1' >> /etc/modprobe.d/qemu-system-x86.conf 

Now, with that out of the way, we can move to the next step

2- Enable nested virtualization in the config of the virtual machine, either with virsh edit or edit the file manually and reload it, whatever you are used to doing should work

virsh edit androiddev

Now, specify either host-model OR host-passthrough, host model is more compatible when moving the virtual machine to a new CPU, while host-passthrough will deliver absolutly all CPU features to the guest os, but is very unfriendly to moving the machine to a different KVM host

<cpu mode='host-model'> 

What services are listening on what ports ?

netstat -tunlp
or the all new ss which shares the same command options
ss -tunlp

for a better view of the files ! (Depending on your setup, you might be able to know which php script is generating outgoing traffic)

lsof -nP -iTCP -sTCP:LISTEN

In this mini tutorial, I will be adding the user kareem to the system, and allow kareem to sftp into a web directory where he can post his web design work, as usual, the steps first, then whatever explanations !

There are two ways to do this, one to add one user, the other to add a group of users, you can either pick one, or do both !

The part in common between both solutions

apt-get install openssh-server
adduser kareem
Then enter a new password twice for kareem

The interesting thing about this sftp user business is that the directory we will specify as the root for the user kareem has to be owned by root ! so go ahead and create the directory /var/www/html/usr/kareem, then execute the following commands

chown root:root /var/www/html/usr
chmod 755 /var/www/html/usr

chown kareem:kareem /var/www/html/usr/kareem

Now, the user kareem owns a directory within his root directory that he can write to, and can not write outside that directory since he does not have the OS permissions, Now, let us add kareem to the list of people who have sftp access but not ssh access.

Edit /etc/ssh/sshd_config and append the following to the document

Match User kareem
ForceCommand internal-sftp
PasswordAuthentication yes
ChrootDirectory /var/www/html/usr
PermitTunnel no
AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no

Now, restart the service by executing the following command

systemctl restart ssh

You are done, try connecting with something like winSCP

Besides winSCP, you can also simply mount the linux filesystem where you have permissions on your windows machine, here are the complete instructions on how to do that https://www.qworqs.com/2022/10/09/mounting-a-remote-linux-file-system-as-a-windows-drive/

The answer to this should be simple, I initiated the test with

badblocks -nsv /dev/sdb

, first, interrupt bad blocks with ctrl+c, the output should be

Checking for bad blocks in non-destructive read-write mode
From block 0 to 1953514583
Checking for bad blocks (non-destructive read-write test)
Testing with random pattern:   0.92% done, 49:38 elapsed. (0/0/0 errors)
 21.32% done, 18:49:24 elapsed. (0/0/0 errors)

Interrupted at block 416437376

Interrupt caught, cleaning up

Okay, so we know what blocks it was supposed to check (1 through 1953514583), and where it was interrupted (416437376)

So i will ask it to resume testing from where it finished (-1), up to the end

badblocks -nsv /dev/sdb 1953514583 416437375

n = Non destructive
s = Show progress
v = tell us about what you find !

The new run should tell you the percentage correctly, but the time counter will be reset to zero, as it is only counting how long this run has been running for

One thing to note is that bad blocks can be used to instruct the filesystem to avoid the bad blocks, but it also allows the disk’s firmware to substitute bad blocks with spare blocks, so that the disk works again with no intervention from your end !

So for my 2TB hard drive…

416437375 = 21% (13 hours)
619014719 = 31.6% (+23:22)
627995199 = 32.15% (+1:04)
667782398 = 34.18% (+4:46)
715469885 = 36.62% (+5:44)
827834875 = 42.38%

While running the tests, you might want to keep an eye on the hard drive temperature with a command like

hddtemp /dev/sdb

To create a log file of the bad blocks, every run should have it’s own file !

badblocks -nsv -o /root/badblocks3.txt /dev/sdb 1953514583 627995198

The concatenation of those files you are creating is very useful in creating a file system if you ever decide to format the drive later !, but the recommended way is using badblocks with the other disk tools directly

while the test is running, you will see 3 numbers that correspond to readerror/writeerror/corruptionerror

Superior in both Lossless compression, and Lossy compression, webp is the new image format by google

Already supported by all web browsers *(that i have tested it with), webP is indeed a promising format, so let us get to compressing our images

I have a big bunch of bitmaps that my scanner spits out (To avoid lossy jpeg compression the scanner’s driver produces), and i need them converted to lossless webp to save space (the first image I compressed went from 552MB bitmap to 183), that is 33% of the original size

So, under linux, this is how i would convert all BMPs into webp images, I think it is exectly the same on windows

on the command line, the command for compressing one image looks like

cwebp -lossless 00.bmp -o 00.webp

Now, the next step is to run them in a batch, copy the following text into a file and name it with the extension

On windows, you might have been using text editors that search or search&replace within files in a folder, one such tool i have used in windows is “source edit” by Joacim Andersson (Brixoft Software). that text editor does not seem to be maintained any longer as the developer seems to have moved into making games, but there are certainly many other editors that allow you to do the same thing.

On the other hand, on Linux, I don’t need to do that, the basic tools that come with the operating system allow for that, multi gigabyte files can be searched and have certain text replaced at the speed it takes to read them (Without having to open them for editing)

So, let us assume we have a folder with many text files (Including css or js or html or php files for example), to search that folder, we can combine

grep -Ril "text-to-find-here" /path/to/file/

-R (-r) look for files recursively
-l show file names, not the contents that were found
-i ….

Another tool which is better suited for looking in code is ack (ack-grep) which i will come back to cover in this article, and a newer tool that i have never used is

Now, replacing a string inside a file is simple, there is a cool tool called sed

sed -i '/TEXTTOFIND/ s//TEXTTOREPLACEWITH/g' verylargefile.txt

Now, to find all occurances of a string in all the files in a directory and it’s subdirectories, you can use the following command, Mind you, this is always treated as a regular expression, if your string contains a dot or anything else that is part of regular expression syntax, you will need to escape it, to avoid that, check out the sd command below

find -type f -exec sed -i 's/find/replace/g' {} +

{} + invokes the “exec” command with multiple file names at once, instead of once per file

The sd command

the -s flag disables regular expressions, sd and fd have rust crates, apt install fd-find sd

fd --type file --exec sd 'Find' 'Replace'
Or with backup
fd --type file --exec cp {} {}.bk \; --exec sd 'from "react"' 'from "preact"'

-s : No regular expressions

In some cases you can even forget about fd, as sd is now capable of dealing with multiple files
sd -i "\n" "," *.txt