Author: Alan

Certbot automatic authentication

Enable certificate auto renew after a manual renew.

I have a number of websites run from my own web server, like this one. Something I set up to experiment with web technologies and gain some insight into how things work.

One of the things I did was setup HTTPS for the websites once I found about about EFF‘s LetsEncrypt service. I wanted to see if I could provide secure connections to my sites even if they’re only for browsing.

I was able to get HTTPS working for my sites and have the certificates renew automatically. Then I changed ISPs. With TWC, now Spectrum, there was never a problem with the automated renewals. With Optimum the renewals didn’t work.

Emails alerting me to certificate expiration were my first indication there was a problem.

The logs indicated that files on my server couldn’t be manipulated to confirm my control of the website. Plus, entering the website address as boba.org or http://boba.org no longer connected to the website (externally, on the local network it still worked). Connection to any of my hosted sites now required prefixing https:// to the name. Automatic translation from http to https no longer worked.

After talking, chatting online actually, with Optimum they told me yup, that’s just the way it works. “We block port 80 to protect you” and “you can’t unblock it”.

Panic! How to maintain my certificates so https continues working? Fortunately certbot offers a manual option that requires updating DNS TXT records. It’s slow and cumbersome and NOT suitable for long term maintenance of even one certificate containing one domain but it works.

Sixty days pass and the certificate expiration emails start again. This time I determined that I’d speak to a person at Optimum and not use the chat. After some time with my Optimum support tech, and after she escalated to a supervisor, I was told there is in fact a way to open port 80. And it is a setting available to me via my account login. So I opened port 80 and thought all set now, renewals will happen automatically.

Not so. I got more certificate expiration warning emails. What to do? All the automated renewal tests I tried indicated a problem with a plugin. I read the certbot documentation, did searches for the error and tried to find a solution that was applied to the problem I had. I didn’t find it. But I did get a clue from a post that said once a manual certification has been done that setting needs to be removed before automated renewal will work again.

After more digging I discovered the certificate config files in /etc/letsencrypt/renewal. In them were two variables that seemed likely to be related to the auto renew problem. They were authenticator = and pref_challs =. The settings were manual and dns-01 respectively.

I never touched these files. It turns out doing manual renewal with DNS TXT records using the command sudo certbot certonly --manual --preferred-challenges dns --cert-name <name> -d <name1>,<name2>,etc just changes the config files in the background. Attempting auto renew later doesn’t work because the settings in the config files have now been changed to authenticator = manual and pref_challs = dns-01.

There was no help I could find that explicitly listed the acceptable values for these variables. And I didn’t have copies of these files from before the changes. After digging around in the help for a while I decided it was likely they should be authenticator = apache and pref_challs = http-01.

I made the change for one certificate and tested auto renew. Eureka, it worked!!

Next I changed the config files for all the certificates and did a test to see if it worked.

$ sudo certbot renew --dry-run
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates below have not been saved.)
Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/alanboba.net/fullchain.pem (success)
/etc/letsencrypt/live/andrewboba.org/fullchain.pem (success)
/etc/letsencrypt/live/danielboba.org/fullchain.pem (success)
/etc/letsencrypt/live/kevinkellypouredfoundations.com/fullchain.pem (success)
/etc/letsencrypt/live/www.anhnguyen.org/fullchain.pem (success)
/etc/letsencrypt/live/www.conorboba.org/fullchain.pem (success)
/etc/letsencrypt/live/www.mainguyen.org/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)

It worked. All my certificates will again auto renew.

This website was created after the problems began. So I didn’t even attempt to make it https. Now that I’ve figured out how to have my certs auto renew again I’ll be converting this site over to https too.

Windows 10 images

Windows’ various versions are the computer operating system I’ve supported my entire professional career. There have been very occasional instances of supporting other systems like Mac’s OS, both before and after Apple switched their OS to UNIX.

There’s many things I don’t like about Windows. I’ve stopped using it for my personal systems for around a decade now. One of many gripes is the installation and update process.

For a while I was fortunate enough to have a professional staff who developed Windows deployment images for our company. They were very good and made image deployment “just work”. It was to the point that about all that was necessary was network boot the pc, point it to the image source and sit back and wait.

I reviewed the procedures they created. Asked questions to better understand what needed to be done to create the Windows images. I never actually was hands on creating an image though. Not from my staff’s documentation and not with any of them shoulder surfing me through the process.

Years later I reached the point of needing to create zero touch deployment images on my own. I failed. It seemed I was close to the solution but never quite there.

Microsoft’s documentation is terribly frustrating for me for the task of image creation. I’ve not found a single Microsoft webpage that goes from zero to bootable deployment image. There’s lots and lots of webpages with instructions for various portions of the work. And some webpages with basic outlines that have links (too many) to details that themselves have many links to more details. Alice never went down such a deep rabbit hole.

Then I found Kari Finn`s guide to “Create media for automated unattended install of Windows 10” on tenforums.com. Kari takes all the diversions Microsoft provides and narrows them down into a single linear process that goes from having installation media to having a zero touch custom installation image. BRAVO and thank you Kari!

Using the guide I’ve finally made my first successful zero touch deployment image!!!

From here I’ll make custom images for the software installations and architectures, BIOS/MBR and UEFI/GPT, that I need to support.

Finally I can make my own images. The world is my oyster.

About time

I am fascinated by science. It defines, refines, and changes our understanding of our very existence.

Changes in understanding both great and small have included insects and mice springing spontaneously from rotting food and undisturbed left over grain, the earth going from the center of the universe to just a planet orbiting a star in a solar system orbiting a galactic core, and the sun being the only star in the universe with planets to stars with planets being common.

To follow a scientific method everything that’s defined must be observable and repeatable by independent researchers.

So how are things defined that aren’t testable? Things like time. Wait, what? We all know what time is, yes. We experience the flow every day. Our watches measure it. Our phones, computers, and networks measure it to enable communication. Scientists predict eclipses and tourists flock to see them based on the predictions the scientists publish. It all seems to work.

Then you read something like this, Do We Actually Experience the Flow of Time?

Now what.

Passwords: Make it safe

Got hacked, locked out of files and accounts? It happens to lots and lots of people.

A few people are actual selected targets. A small minority I believe. The others? They’re the “catch” the result of cyber criminals casting a wide net with their tools.

When I talk with people about safe passwords they often say things like “I can’t remember so many” or “It’s too hard to come up with good memorable passwords” and often “I just don’t understand how to manage it”.

To them I say a password manager is your friend and protector. Refer to this article, Why You Shouldn’t Use Your Web Browser’s Password Manager, for useful information about password managers.

A few things that I see a bit differently than the article.

First, I disagree with the basic premise. In my experience the best way for people to start doing something new is to start from where they’re already at.

So if you want to use the password manager in your web browser then go ahead. You must stick to using that browser. If you already do that why not stick with it?

Second, I disagree that the open source password managers mentioned are more complex than the password managers mentioned, especially if you already store files on the cloud with Dropbox, Google Drive, etc.

And I see an advantage for the separate password manager. If you use a password manager like KeepassX and the file sharing site gets hacked you’re still the only one with the password database’s password. If your online password manager site is hacked then all your passwords are compromised.

In the case of using your browser or something like LastPass to manage your passwords an account must be created with the provider of the password management service. Essentially only one layer of protection.

If KeepassX or something similar is used there’s two layers. The file sharing website and the password database itself.

Multi layer protection is where it’s at baby! (said in Austin Powers voice)

The most important part of all this is to set up different complex passwords for each site you use.

Use your browser’s password manager, an external service like LastPass or a separate password manager like KeepassX combined with an online file storage service to create unique complex passwords for each site you use and you’ve improved your security by leaps and bounds.

Virtual Host??

Setting up Apache to support multiple websites on one host. My server already does that for my public websites.

However I want to control what is returned to the browser if a site isn’t available for some reason. So I’ve set up a virtual server with multiple sites. Each site works when enabled. However if the site is set up to be unavailable, disabled, no index file, etc. the default page returned to the browser is not what I’d like.

Need to identify a few fail conditions, see what the server returns when the condition exists, see if what’s returned for a given condition is the same regardless which site the failure is generated by, then figure out why the webserver is sending back the page it does.

Reasons not available:

  • site not being served, e.g. not enabled on server
  • site setting wrong, e.g. DocumentRoot invalid
  • site content wrong, no index file

Answers that might be returned:

  • site not available
  • forbidden
  • …other’s I’ve seen but don’t remember now

From what I’ve read it seems whatever’s in 000-defalut.conf should control which page/site loads when a site isn’t available. That’s not the result I’m getting.

Either I’m doing it wrong or I’m just not understanding what’s supposed to happen and how to make it happen.

More digging…

VBoxManage

Important VirtualBox command to be familiar with. Get virtual machine info that can be copy pasted into documents and other commands.

vboxmanage list runningvms

Also display running machine properties without having to navigate the UI. Good for quick review of network settings too.

vboxmanage showvminfo "VWebHostTest" | grep "Name: \|Rule"
Name:                        VWebHostTest
NIC 1 Rule(0):   name = Web8000, protocol = tcp, host ip = , host port = 8000, guest ip = , guest port = 8000
NIC 1 Rule(1):   name = Web8001, protocol = tcp, host ip = , host port = 8001, guest ip = , guest port = 8001
NIC 1 Rule(2):   name = Web8002, protocol = tcp, host ip = , host port = 8002, guest ip = , guest port = 8002
NIC 1 Rule(3):   name = Web8003, protocol = tcp, host ip = , host port = 8003, guest ip = , guest port = 8003
NIC 1 Rule(4):   name = Web8004, protocol = tcp, host ip = , host port = 8004, guest ip = , guest port = 8004
NIC 1 Rule(5):   name = ssh, protocol = tcp, host ip = , host port = 2223, guest ip = , guest port = 22
NIC 1 Rule(6):   name = web8080, protocol = tcp, host ip = , host port = 8080, guest ip = , guest port = 8080
NIC 1 Rule(7):   name = web8800, protocol = tcp, host ip = , host port = 8800, guest ip = , guest port = 80