Sometimes you need to find the largest sub directory in a directory, for this, the DU command has an argument that can work for you, max-depth
du -h --max-depth=1 backup-final
The above will simply list all sub directories inside the backup-final directory alongside the size of the directory (recursive, including all files in sub directories)
One of the most annoying things that can happen to you is to disconnect your laptop from the network, then realize that there was a file moving job that was running, the command is going to get disconnected before mv gets the chance to delete the source files, the new copy is a hassle, i personally use rsync to continue such a copy with the delete source files flag
But it does not have to be this way, you can move the mv command to the background, the steps are simple
First, you need to hit CTRL+z to suspend the job, once suspended, you should see a job number in the suspend acknowledgement
Now, the next step is to disown the job, because right now, if you close your terminal window, the job will still be terminated
disown -h %1 (replace one with your own job number which may be 1)Now to get the job running again but in the background
bg 1That is it, you can now close (logout) your terminal window
So the summery ctrl+z disown -h %1 bg 1 logout
Now mind you, the output to stdout will not display (In most cases), you will need to use process status ps x to see the process.
If you want to bring back the command into the foreground, all you need to do is execute the command jobs (to find the ID), then run the command fg 1 (if it was job number one), then to hide it again, you can ctrl+z then bg 1 again (No need to disown it this time)
Sometimes, you realize you would want to use LVM in a computer but you want a tutorial to take you there fast, So i have put this post together to get you up and running in 10 minutes or less.
First, let us start with the tools, install the following packages in your debian installation, (Debian 10-Buster in this tutorial)
apt-get install lvm2 dmsetup mdadm reiserfsprogs xfsprogs
In this tutorial, I have two 8TB disks i want to combine with LVM to use as one
Start by running parted on every physical disk, and creating a big partition to span the whole disk (Or if you are using empty spaces on old disks, any partition is good enough really)
In my example, where i wish to combine /dev/sdb and /dev/sdc into one logical volume (To be used as one block)
parted /dev/sdb
mklabel gpt
unit mib
mkpart primary ext4 1 100%
set 1 lvm on
Now, repeat the above, but this time for /dev/sdc
1- Prepare the physical volumes to be added to volume groups (those two partitions) (For instructions on how to add disks directly without an underlying partition which is not recommended, see here)
pvcreate /dev/sdb1 /dev/sdc1
Now, you can see what we have with the commands
pvdisplay or pvs
vgcreate LVMGroup /dev/sdb1 /dev/sdc1 Volume group "LVMGroup" successfully created
this will create a volume group (VG) called LVMGroup, you can see the VGs you have with the commands (vgdisplay, vgscan, and vgs)
vgs -o +lv_size,lv_name
lvcreate -l 100%FREE -n bigvol LVMGroup Logical volume "bigvol" created.
mkfs.ext4 /dev/LVMGroup/bigvol or mkfs.ext4 /dev/mapper/LVMGroup-bigvol
To mount the volume, My mount point is at /hds/lvm
rsync is better than mv to move files because of a few reasons
First, it gives you much more control, for example the following command
rsync -a -v --ignore-existing --remove-source-files /hds/iscsi/all_new /hds/usb
Does not copy the files that are already at destination, meaning if there is a file with the same name in the same directory at the destination, it will not be overwritten, files moved will be deleted, and files that had counterparts and not moved will not be deleted
rsync -a -v --remove-source-files /hds/iscsi/all_new /hds/usb
While the command above will overwrite files, and delete whatever we have moved ! if files exist on the destination, it does not seem to be overwriting them, but it is probably somehow comparing them, then deleting the original
I had to investigate this as a move command resulting in an error did not remove the files, the error was relevant to the file name being too long
When tarring a file, even if the process (Nice) priority is low, sometimes the IO will make the computer unusable, so what you can do it ask Linux to only work on the IO intensive job when there is no other application requiring IO
ionice -c 3 -p $PID
The above class (3) means only if no other application is using this disk
The answer is, depending on your needs, but you are not here to hear that now are you, so I did a little experiment, tailored specifically for a certain need that does not apply accross the board, but combined with other experiments online, this might give you an idea
The experiment is annotating an image with image magic plugin for PHP
Both running buster (Debian/Raspbian 10), both on PHP7.3 and everything fresh installed
the PC has 8GB of DDR3 Ram, and the PI 1GB of LPDDR2-900 SDRAM
The PC has a 12 year old “Intel(R) Core(TM)2 Quad CPU Q9450 @ 2.66GHz”
The PC was around 7 fold faster than the raspberry PI 3 (the mean of a trial of 20 runs, with all the files cached in ram by the linux kernel)
The trial was done with a discrete VGA card (AMD RADEON) and with the chipset’s own graphics card, results were identical (The VGA card did not have it’s drivers installed, so we don’t know if it might have an effect or not)
I am now moving into even older PCs to compare performance, and will report the results here again to see what the limiting factors might be
Well, to make a long story short, I make large scans that don’t need to be on my system disk, I would like to have them stored on the spinning disk, So, i searched online, and found that people were suggesting that I create a symlink to the new location in place of the Scanned Documents folder, Something like
mklink /d "C:\Users\yazeed\Documents\Scanned Documents" D:\scanner
The above obviously has downsides, backup software, changed disk IDs, or other unforeseen effects, In my mind, there is no way the developers of Windows Fax And Scan have hard coded this into the application,
Another solution would have been to move the entire Documents folder (Right click it, then change it’s location, windows provides that option), But not everyone wants that, So there is a third more elegant solution (Even though this works like a charm)
The third is to look in the registry (Find) where Windows Fax And Scan stores it’s path (Look for the word scanned for example), and change that, but i Have already gone with the second, the third should be simple
The lowdown, applying grease to the fan pin works, no idea for how long (Already ordered a replacement fan from the HP website just in case), but when this fan fails, I will certainly update this post (Today is April 18, 2020).
The only reason i use my HP Envy 17-s143cl which has a slow I7-7500U (X0S43UA) is because it is silent most of the time, and quiet when the fan eventually spins up, not any more, today, the fan was very loud, and vibrates heavily, sometimes banging above the fan resolved it (Don’t try if you have a spinning disk, you will damage it, I have an SSD inside so it is okay).
So i took the laptop apart, extracted the fan, opened the fan casing, and then the plastic fan separates from the coil by simply pulling it out (With very little force, if you need any force you are doing it wrong).
So, like you see in the photos above, I applied some automotive lithium grease to the pin, and put everything back how it was, the fan is working very smoothly, just like before, if it fails and that is a good possibility, I will update this post, but for now, it looks like a hit
Also worth mentioning is that i usually add such lubricant to PC fans, the cheap $1 fans, they usually need the lubricant after a few months, but after adding the lubricant under the back sticker on a desktop PC fan, they usually work for years more without problems, so I expect this to not give me any trouble, but in any case, I have already ordered a replacement just in case, turns out the HP store is cheaper than ebay and amazon, Just go to HP and find the part you need, mine was the one below. it cost me with shipping an all around 32 US dollars.
806747-001 In stock
FAN – for Broadwell SKU (Fanless design for Braswell SKU)
RoHS: COMPLY_2.05
| Order subtotal | 18.26 |
| Shipping and Handling | 11.25 |
| Tax | 2.62 |
| Order total (USD) | 32.13 |
In short, You can (Up to date systems) mount it in 1 go with something like
mount -o loop /path/to/my-iso-image.iso /mnt/iso
Step 1, what is the next loop number we can mount on ?
losetup -f
This will probably result in something like /dev/loop0 (Which is the next ready loop to map the ISO to
losetup /dev/loop0 /path/file.isoThen, mount the loop device with the mount command
mount /dev/loop0 /mnt/dirThat’s it, The new solution and the old, As always, i keep things here as a not to self, because sometimes I come across older computers, and I forget how it was done since all the mounting I do now the new way
In this post, I will be explaining how to watermark videos with a PNG image watermark that is transparent where it needs to be, I will cover both Linux and Windows (Not much is different on the ffmpeg side, the difference is when you want to traverse a directory (The script).
The watermark you see here is what I want to overlay over the video, If you right click and view the image, you should be able to see that around the text, it is transparent.
now, let us assume that the file in the directory is called x.mp4, and this watermark image is called watermark.png, then the following commands should overlay this image over the video
ffmpeg -i x.mp4 -i watermark.png -filter_complex "overlay=10:10" x1.mp4
The code above will create a new file (x1.mp4) which has the overlaid watermark, as you might be able to see if you execute the above the watermark is positioned at the top left corner of the video, which is not necessarily what you want, now because we know the dimensions of the watermark image, we can ask ffmpeg to center it horizontally (and if you like vertically to have it in the center of the image, but this is not what i want.
So let’s assume the video is full HD, meaning it has the dimensions 1920 x 1080 (Width x Height), and the image, as you can see has the dimensions (500 x 100), what i want here is to have the watermark centered horizontally and nudged down 100 pixels vertically, the code to do that would be
ffmpeg -i z.mp4 -i watermark.png -filter_complex "overlay=x=(1920-500)/2:y=100" z3.mp4
And in case this is not clear, here is a code to place it in the bottom right side of the screen
ffmpeg -i z.mp4 -i watermark.png -filter_complex "overlay=x=(1920-500):y=(1080-100)" z6.mp4
Now with the process of watermarking out of the way, How do we batch process videos under windows and under linux ?
Under Linux it is simple, I put all the input files in a directory named “in” and all the output is to be put in an directory called “out”, the shell script (batch file) is at the root where those 2 directories exist, the shell script is this
!/bin/bash OIFS="$IFS" IFS=$'\n' for filename in "in/"*.mp4; do ffmpeg -i "$filename" -i /apth_to_watermark/watermark.png -filter_complex "overlay=x=(1920-500):y=(1080-100)" "out/$(basename "$filename" .mp4).mp4" done IFS="$OIFS"
I have never been good at windows, so i looked around for a script to traverse a directory, I found some stuff, and here is my final result, if you can clean it up and make it more robust, please do leave me a comment and i will improve with your recommendations.