Differences

This shows you the differences between two versions of the page.

--- quickreference:unix [2023/10/08 14:43] – rodolico
+++ quickreference:unix [2025/02/05 00:12] (current) – [Shell (mainly BASH)] rodolico
@@ Line 5: / Line 5: @@
 ===== Systems Administration =====
-==== Rapidly wipe multiple hard drives ====
+==== Partitioning large drives ====
-Nothing beats DBAN [https://dban.org/] in ease of use and a feeling of good security. However, I recently had an issue where I had a server with 7 slow hard disks containing data that really wasn't all that sensitive, so I simply wanted to put a bunch of zeros on it, so I booted off of my SystemRescueCD thumbdrive [https://www.system-rescue.org/] and ran the following commands.
+Drives greater than 2 Terabytes are not handled well by the standard //fdisk// application, so instead we use parted. Fun Fact!!! gparted is a nice little GUI interface to this. But, we're dealing with command line stuff here.
+This assumes we have a drive, sdg, that we want to set up with gpt and create one partition on. That partition will set up on optimal sector boundries, and use all of the space available.
 <code bash>
-# for truly not sensitive information, this command wipes all the OS information
+# remove all old file system information. Not necessary, but I do it just because I can
-for drive in a b c d e f g ; do wipefs -a /dev/sd$drive ; done
+wipefs -a /dev/sdg
-# but, to really remove in a way that takes tons of effort to recover, do this also
+# make this a gpt disk. Will wipe out any other partitioning scheme
-for drive in a b c d e f g ; do echo Cleaning sd%drive ;  dd if=/dev/zero | pv -petrs 580G | dd of=/dev/sd$drive ; done
+parted /dev/sdg mklabel gpt
+# make a new partition on optimal sector boundries. This is a primary partition, and starts
+# at the beginning of the disk (0%) and goes to the end of the disk (100%)
+# I put that in quotes as, from what I've read, the percent symbol does not work well
+# within the bash command line
+# note, we are not telling it what file system to use, so it defaults to Linux
+parted -a optimal /dev/sdg mkpart primary '0%' '100%'
+# display the information on the disk
+parted /dev/sdg print
+# format as ext4, no reserved space, and a disk label marked 'backup'
+mkfs.ext4 -m0 -Lbackup /dev/sdg
 </code>
+==== Rapidly wipe multiple hard drives ====
-I then realized I had 7 drives which should be able to be accessed in parallel, and also the dd will wipe all the fs signatures anyway, so I changed my command to.
+Nothing beats DBAN [https://dban.org/] in ease of use and a feeling of good security. However, I recently had an issue where I had a server with 7 slow hard disks containing data that really wasn't all that sensitive, so I simply wanted to put a bunch of zeros on it, so I booted off of my SystemRescueCD thumbdrive [https://www.system-rescue.org/] and ran the following bash script. Should work in any shell which has the //for// command, however.
-<code bash>
+<code bash wipedrives.sh>
-for drive in a b c d e f g ; do screen dd if=/dev/zero of=/dev/sd$drive status=progress ; done
+#! /usr/bin/env bash
+# for truly not sensitive information, this command wipes all the OS information
+for drive in a b c d e f g
+do
+   wipefs -a /dev/sd$drive
+done
+# but, to really remove in a way that takes tons of effort to recover, do this also
+for drive in a b c
+do
+   echo Cleaning sd%drive
+   dd if=/dev/zero | pv -petrs 580G | dd of=/dev/sd$drive
+done
 </code>
-Note: I have not done the parallel one yet. I'm waiting for the previous command to finish, which is estimated to take about 35 hours (5 hours per drive). I'll update this with the results of trying the parallel method when I do.
+I had 7 drives to wipe, and this takes about 5 hours per drive, so a total of 35 hours. I realized I could probably run all 7 processes in parallel since, on my system, the drive controller is a lot faster than any individual drive So I decided to use the //screen// command and see if I could make that work.
-Haven't done this yet. The
+<code bash wipedrives2.sh>
+#! /usr/bin/env bash
-==== Check SSL Cert Expiration Date ====
+for drive in a b c d e f g h
+do
+   screen -dmS sd$drive bash -c "dd if=/dev/zero | pv -petrs 580G | dd of=/dev/sd$drive"
+done
+</code>
-Ever wondered when your SMTP SSL Certificates are up for renewal? A quick and dirty way of doing it from the command line was shown at [https://serverfault.com/questions/131627/how-to-inspect-remote-smtp-servers-tls-certificate#131628].
+Basically, we're using a bash for loop to grab all the drive names (I just used the last letter), running screen and immediately detaching the new process after telling it to run //bash -c// and the command after it in quotes (so it would not interpret the pipes in our current, non-screen shell). I'm running this right now, and //pv// is predicting it will be done in 11.5 hours, or less than a third of the time. BUT, it is really heating up the office with 7 drives being continuously written to at the same time.
-Note: the discussion covered other things, and is well worth a 5 minute read.
+**Warning**: When SystemRescueCD boots, it tries to assemble any mdadm (software RAID) arrays, and since they are locked, //dd// and //wipefs// won't be able to write to them (maybe). In that case, do the following:
 <code bash>
-printf 'quit\n' | \
+# find any mdadm volumes running on Linux
-openssl s_client -connect smtp.example.com:25 -starttls smtp | \
+cat /proc/mdstat
-openssl x509 -dates -noout
+# assuming it showed you md127 was running (normal)
+mdadm --stop /dev/md127
+# it should stop the MD array and make the individual drives accessible
 </code>
 ==== Rename Server ====
@@ Line 50: / Line 84: @@
 <code bash>
 # change the host name, and the postfix name if that is installed
-sed -i.old 's/oldname/newname/g' /etc/hostname /etc/hosts /etc/mailname /etc/postfix/main.cf
+sed -i.old 's/oldname/newname/g' \
+   /etc/hostname \
+   /etc/hosts \
+   /etc/mailname \
+   /etc/postfix/main.cf \
+   /etc/camp/sysinfo-client/sysinfo-client.yaml \
+   /etc/msmtprc
 /etc/init.d/hostname.sh start
 # update the aliases, if they exist
@@ Line 197: / Line 237: @@
 ===== Shell (mainly BASH) =====
+==== Here Documents ====
+Most unix users are familiar with echo'ing something to STDOUT so it can be used as STDIN for a following script (using the pipe). However, this is usually limited to a single line.
+A **here document** is a way of having multiple lines processed at one time. In many cases, you can have similar functionality using quotes, but here documents are more robust.
+For example, a simple test of a newly built mail system might include creating a file with all of the headers necessary, then passing that to //sendmail// for processing. However, you can skip a step and simply send the message directly using a here document.
+<code bash>
+sendmail user@example.com << EOF
+To: user@example.com
+from: root@example.org
+Subject: test
+This is a test
+EOF
+</code>
+The entire block above is one command. Here is the breakdown.
+  - //sendmail user@example.com// is a standard sendmail invocation which assumes the message itself is coming from STDIN.
+  - //<<// marks the beginning of a //here document//
+  - //EOF// is the tag which will mark the end of the text for the here document
+  - Everything up to the EOF is the actual string to be passed to sendmail
+  - //EOF// at the end marks the end of the here document. **Note**: there must be no leading or trailing whitespace. The tag must be exactly as entered after the << (case sensitive), and must be the only thing on the final line.
+This only touches the surface of here documents. See [[https://tldp.org/LDP/abs/html/here-docs.html]] for a lot more information.
 ==== Find files within date range containing text ====
@@ Line 370: / Line 437: @@
   * https://docs.nextcloud.com/server/18/user_manual/files/access_webdav.html
   * https://www.cyberciti.biz/faq/create-a-freebsd-swap-file/
+  * https://www.mybluelinux.com/test-imap-with-telnet/
+  * https://serverfault.com/questions/131627/how-to-inspect-remote-smtp-servers-tls-certificate#131628]