Alan’s Notes – Page 5 – a toe in the water

Chasing my tail and finding something new to learn

Experience and keeping notes helps limit chasing tail.

In my last post, Help people get the job done, I wrote about disappointment with how a change was made in the end user’s environment at my office. The change required they do something different to accommodate a purely technical change in systems. Once connected their work was no different than it had been.

Why we didn’t build in the logic to connect them to the new resource and make it transparent for the user seemed to me like a failure on our part. Simplify the user experience so they can focus on the work they do by IT using our skills to make the computers work for people rather than the other way around.

I made some changes to personal websites to demonstrate redirection could be used to point at the correct work websites. It was meant to illustrate the analog idea that one work website could be pointed at the other. Going to my websites, train.boba.org and sclc.boba.org, immediately sent a browser to the intended work website. Success!

After demonstrating the capability I disabled it so my URLs go to their originally intended websites.

So where’s chasing my tail come in?

While experimenting with the redirect I modified the boba.org configuration. For a while it wasn’t possible to get to that site at all. Then depending on the URL got to it or andrewboba.com. Putting boba.org in the browser’s address bar ended up at andrewboba.com, but not correctly displayed. Putting http://boba.org went to the correct site but didn’t rewrite the link as secure, https://.

To stop being distracted by that issue and continue testing the redirect I disabled the boba.org website.

Worked more with the redirect over a few days. Got to the point I felt I understood it well and tried boba.org again.

It wouldn’t come up no matter what I tried. Everything went to a proper display of andrewboba.com.

I increased the logging level. I created a log specifically for boba.org (it didn’t show up which was my first clue). Not seeing the log I went through other site configurations to see how their custom logs were set up. They appeared to be the same.

Finally I decided to try boba.org without a secure connection. I wasn’t sure the name of the .conf file for secure connections and decided to look in Apache’s ../sites-enabled directory to see if there were separate .conf files for https connections.

And guess what I found? There are separate .conf’s for https, yes. There were no .confs of any kind for boba.org! Then it hit me. There had been no log files for boba.org because there were no ../sites-enabled .conf files for boba.org.

And then I finally remembered I had disabled the site myself to focus on the redirect. Chasing my tail because I’m very new at Apache webserver administration. I disabled a feature to focus on making something happen then forgot the change I made when I resolved the first challenge.

Better notes, and more experience, would have helped me remember sooner.

And I also found something new to learn. While boba.org was disabled, andrewboba.com was being displayed. Would prefer “not found” or something similar to show up rather than a different website on the server.

New challenge. Figure out how to serve a desired site/page not available message when a site on this server is down.

One of the reasons I like information technology. Always something new to learn at every turn.

Help people get the job done

IT’s job is supposed to be making things easier for users.

Users have been using a single URL for access to all their web applications and now the backend for just one is moved to another server to avoid end of life? If you’re where I am now users are sent a new URL and told to use it if that application is needed.

It is accessed via Citrix and I don’t understand Citrix architecture well I have to say. However the users of this app apparently don’t use any other app via Citrix.

In the meeting about the change I wondered out loud whether users could just be redirected? No need to learn a new URL, no need to know when or if to use it. Just send the apps’ users to the new URL when they attempt to use the app.

The response was, “no, can’t do that”, “don’t have wild card certificates”, “can’t install existing certificates on other servers”, “can’t change DNS”, “can’t send people from the old site to the new site”, and so on…

My reasoning was to simplify the user experience. Why make people learn something new if there’s a way to get them to the new webapp without learning a new URL? As a technologist I feel VERY strongly my job and the job of others like me is to enable people to do their work and not force them to understand or learn technology that is not relevant to that.

Back to the objections. A DNS name can have its network address updated periodically. This very website has a dynamic address and can still be found by name even after an address change. The server is running a job to monitor the public address and update DNS when it changes. Automatic. Hands off.

No certificate changes required. If siteA and siteB are continuing to operate as siteA and siteB and each has their own valid certificate then no change in certificate needed. When someone browses to the site the browser requests a secure connection. The trustworthiness of the connection is determined by information the site provides and certificate authorities the browser trusts. No need to move certificates anywhere. Even if there were that can be done without renewing certificates.

Sending people from one site to another, in its simplest (as far as I know) form only requires a Redirect. For wesiteA and websiteB, if visitors to websiteA should actually be going to websiteB tell websiteA’s webserver to redirect browsers to websiteB. When somebody browses to websiteA the webserver sends a message back to the user’s web browser which says you need to ask for websiteB instead. Then the browser does just that and ends up at websiteB even if it’s on a different server in a different country.

I actually set up Redirect on this server to test my understanding and be certain it would work the way I thought. It did. Visiting one of my webhosts on this server automatically directed me to workAppA and visiting another webhost went automatically to workAppB.

In doing the reading to get Redirect set up I learned it could be as granular as by user or program on an Apache server. I suppose it’s possible Citrix doesn’t have a way to support that. But I don’t believe it. I know Citrix apps can be secured by login so userA and userB don’t see all the same apps. I’ve written powershell to report what security groups are associated with which published apps on a Citrix server.

In this case telling end users YOU HAVE TO LEARN SOMETHING NEW to keep doing your job the same way strikes me as IT not doing its job!

Phishing, some examples

A guide to spotting email that is meant to deceive you.

Recently I received a number of phishing emails and shared some with family and friends so they could see examples and hopefully avoid any they might get.

After doing that I decided it would be good to share here too. And I went a bit further and made some (admittedly crude) videos to spotlight some of the indicators that an email is phishing.

The videos are posted on YouTube and I’ve embedded them here.

These were my first attempts at creating videos with effects and titles. Please try not to be critical of the production quality and instead focus on the information provided. You’ll find it useful if you do.

For those of you who might look and say, “They’re too tiny. I can’t see anything.”, after starting the video click in the lower right hand corner of the video window. It will enlarge the video.

This one was meant to get the victim to open an attachment. I may make a post and video of what happens if the attachment is opened. For the time being know that the video has tips to help identify it as phishing so we know better than to try and open the attachment.

This one claims there’s a problem with your Apple ID and has links that connect with a counterfeit Apple website. If you were to click the links and complete the forms you’d be giving away your Apple ID login information. Again there’s titles and effects to help identify the tells that make it apparent this isn’t from Apple.

Technical support

Technical support. Not quite “Vanilla Sky” level stuff but still thought provoking.

I provide some limited technical support on Apache OpenOffice and LibreOffice forums.

Why not Microsoft Office? Because that’s what I do at work. At home I use, and have used for many years, LibreOffice and OpenOffice. Why? Because they save me money, the support forums are generally more congenial and providing support lets me give back a bit for the value I get.

One of the things I really enjoy about providing support is seeing all the ways people are using software to do things they need to do. Things I would never have conceived of. It is a real eye opener to get a handle on someone else’s requirement. And then very gratifying to help someone meet their need.

One of the recent support requests was for a bowling league score sheet kept in Apache OpenOffice. The requestor wished to have the latest match value always be shown in a particular cell. The league has 32 weeks in its season and for each week the bowler’s score is entered for each of three games. The game total and average are calculated and displayed for each week.

The latest week’s match value always needs to show in a particular cell. The method being used was to update the cell manually each time a new week’s scores were entered.

I came up with a solution, posted it. Then, as I often do, rethought the whole idea. What I realized was the way I designed the solution it would always show the value for the 36th week, the last week of the season, rather than the most recently entered week! Not good.

After some deeper inspection I recognized this happened because the formula to show the score for the latest week actually just checked to see if there was anything in the referenced cell. And it counted a formula in the cell as something, even if the formula displayed nothing.

This needed to be fixed! I couldn’t provide a solution that didn’t work.

After some thinking I realized a formula to show the latest value needed to recognize whether the formula on each row displaying the value was showing a numeric value, a blank “”, or a label “DNB”. DNB, Did Not Bowl, was a label indicating the bowler hadn’t bowled that week.

Again, the season is 36 weeks. Scores are entered week by week. This meant the rows showing score and average were always followed by rows showing nothing. Even though cells in the rows contained formulas, the formula result was “” for each week after the latest one entered.

My original formula was detecting a formula in a cell as something. I needed to come up with a formula that could identify the row before the first row with formula result “”. Finally an idea struck. I could use a function that counted the rows where the formula result is “”. A blank. These are always the weeks of the season that are not yet played. The season is 36 weeks. Subtract the number of blank rows from 36 and that’s the last row with a score. Problem solved!

That formula is…

=INDEX($Sheet1.G6:G41,36-COUNTBLANK($G$6:$G$41),1)

With that part of the problem solved I saw that some formulas I hadn’t touched were returning #VALUE! errors. These were formulas that calculated total pins week by week. These errors were happening because of changes I’d made to the formulas to sum the pins for each week and to produce the pin average for each week.

My fixes created the problem so I was determined to resolve it.

I created what I call a “dynamic formula”. A formula that changes based on where it is in the sheet or what it reads from a value elsewhere in the sheet. It didn’t work. It has been some time since I created such a formula in OpenOffice or LibreOffice.

There’s quite a bit of compatible functionality between Microsoft Office, LibreOffice and OpenOffice. For the most part spreadsheets created in one work in the other without modification. For the most part.

As it turned out, I was creating the “dynamic formula” as if writing it in Excel. Dynamic formulas are one of the things that are a bit different between the Microsoft and OpenOffice/LibreOffice spreadsheets. Once I recognized that, I was on the way to developing a solution.

The below dynamic formula, “the solution”, totals values in a column beginning at a specific row and continuing to the row the formula is in.

=IF(ISNUMBER(I6),DSUM($G$5:INDIRECT(CONCATENATE("$G",ROW())),1,$BB$5:$BB$6),"")

The formula needs to calculate a sum from a fixed starting row to whatever row the formula happens to be in. And in the case of the bowling league it needs to do that for thirty-six rows. If the formula couldn’t tell which row it was in and sum from the first row to the formula’s row then thirty-six different formulas would need to be entered. One for each row.

Entering the same formula in thirty-six rows is much easier in my opinion. And easier to maintain and easier to modify.

By tackling this person’s question I:

Helped solve a problem
Familiarized (again) with the difference between Calc and Excel dynamic formulas
Learned about a process, a “functional requirement”, I wasn’t familiar with and provided a way to support it

For me, this was a win all around. What could be better than the warm glow of finding the solution to a previously unknown use case?

Certified Information Systems Security Professional, CISSP

Security. Human factors are always important.

I hold a CISSP certification. Information security is something I’ve found intriguing since I first started my technology career. One of the first user trainings I developed was around the time of the “I love you” malware that struck via a deceptive email attachment. And to this day email continues to be a vector for compromising systems. Or actually I should say, email account holders continue to be a vector for attacking systems.

My office at the time of “I love you” wasn’t struck by it but we would have been except for our mail system. Everyone in the business, about 160 people at the time, had gotten the system security training. And a special alert had gone out after the training warning of “I love you”. By and large the people in the company were well educated professionals with uncommonly high expectations around privacy and confidentiality. Our work was providing counseling and permanency for youth and families involved with various states’ child and family services departments.

What I mean to say is the staff of the organization all understood and practiced privacy and confidentiality and so were an interested and engaged audience for the security training.

With the above as background, this is the story of “I love you” in my office.

One day the Executive Director’s Administrative Assistant called me and said, “Alan, I think I’ve done something I shouldn’t have.” She explained she had gotten an email from the building’s manager with an “I love you” attachment. The man was someone she dealt with often and was on good terms with. She was married and was a bit upset by getting an email with such a bold attachment. She was also intrigued wondering why he would send it to her and what message might be inside.

She didn’t delete the email immediately but kept it and wondered what message it might contain. Finally she opened the email and attempted to open the attachment. Nothing happened.

Our mail system was Lotus Notes client and server. The malware relied on Visual Basic Scripting in Microsoft Outlook and so was unable to propagate in our environment.

This is a case where a knowledgeable person with a commitment to privacy and confidentiality and who had gotten security training as well as read the follow up warnings about “I love you” nearly caused a security incident because of curiosity! The only reason there was no incident was because of a technical feature of our environment.

She realized something was wrong when there was no message to see. And then she relied on her training, called me, and confessed to maybe doing something wrong.

This is a lesson that’s stayed with me. You can have good people and good training but good technical measures are still needed to back them up. People will occasionally do things they suspect might not be in their best interest because of some other overriding impulse, like curiosity.

And this brings me to something else, earning CPE (Continuing Professional Education) credits to keep my CISSP current. I generally enjoy the briefings and learn many interesting things while earning CPEs. However I do struggle sometimes because it is difficult at times to find CPE courses that are not too strongly vendor centric. My preference is for training that is less about the knobs and switches of a particular technology and more about the ideas behind threats and countermeasures.

I was really pleased to get a mailing from (ISC)2 the other day. It introduced courses that are free for members that providing training and CPEs. Much of the training looks to be very relevant to my interests and I’m very excited to get started!

Courses like:

Techniques for Malware Analysis
Web Appliction Penetration Testing
Gaining Support for Your Security Program
Introduction to NIST Cybersecurity Framework

…and others are all about topics that I expect to be quite enjoyable.

I also will be producing another post with some examples of phishing attacks I’ve received. Some that were quite good and nearly motivated me to reveal credentials.

Certbot headaches!

Modifying certificates with certbot. It works and it was a long journey to get it done.

If anyone’s reading this you may have noticed the URL is wp.boba.org. Possibly you entered www.wp.boba.org to get here and saw it change to wp.boba.org. Whatever. Until this evening (15 Jan, 2020) the URL’s protocol would have been http://. https:// wouldn’t have even connected. Now, even if http:// is entered it changes to https://. Hooray!!

The SSL certificate for this website is now part of the alanboba.net certificate. But, until tonight, I was unable to expand the domains in the alanboba.net certificate to include wp.boba.org and www.wp.boba.org.

My attempts to expand the alanboba.net certificate began nearly a month ago. Everything I tried failed. In desperation I posted on the LetsEncrypt community forum a little over three weeks ago, Apache certificate modification not successful, hoping someone would quickly recognize the problem and suggest a solution.

That didn’t pan out. Not a lot of respondents. The fix suggested didn’t address the issue the error message presented, the request was “unauthorized”, or suggest if the message might be misleading.

Domain: wp.boba.org Type: unauthorized Detail: Invalid response from http://wp.boba.org/.well-known/acme-challenge/QVV-1Skk-Xvrr6QAL-IvvDZuMGnhr2mNOfoAWbkYCnw [67.86.147.116]: "\n\n404 Not Found\n\n

More reading. More checking settings on this server. Some experimental configuration changes to see if the issue resolved and the command certbot –expand… would succeed in adding two additional domains to the existing certificate. None of the changes worked.

Finally came across a different command and decided to try it. As I understood it, it is meant to renew existing certificates not add domains to them. However it does include a “webroot” parameter and some of the documents I’d read suggested the webroot location might not be correctly interpreted by the command I was using.

The documentation I found doesn’t say anything to suggest the command can be used to expand the domain names covered by a certificate. I just had an inspiration and decided that if webroot might be the problem then explicitly specifying the webroot and adding domain names at the same time might turn the trick.

Tonight I tired the command with the webroot parameter and my additional domains appended to the list of domains already on the certificate. Surprise and delight! The domains were added to the certificate AND the protocol is now changed to https:// even if http:// is used in the URL name!

The following command…

sudo certbot run -a webroot -i apache -w /var/www/wp.boba.org/public_html -d alanboba.net,boba.org,sclc.boba.org,train.boba.org,training.boba.org,www.alanboba.net,www.boba.org,wp.boba.org,www.wp.boba.org

Produced the output below. Plus it added my two additional domains to the existing certificate and modified apache’s config for the website so http:// requests are rewritten as https://. Like I said at the beginning, hooray!!

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer apache
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
You have an existing certificate that contains a portion of the domains you requested (ref: /etc/letsencrypt/renewal/alanboba.net.conf)
It contains these names: alanboba.net, boba.org, sclc.boba.org, train.boba.org, training.boba.org, www.alanboba.net, www.boba.org
You requested these names for the new certificate: alanboba.net, boba.org, sclc.boba.org, train.boba.org, training.boba.org, www.alanboba.net, www.boba.org, wp.boba.org, www.wp.boba.org.
Do you want to expand and replace this existing certificate with the new certificate?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(E)xpand/(C)ancel: E
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for wp.boba.org
http-01 challenge for www.wp.boba.org
Using the webroot path /var/www/wp.boba.org/public_html for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/sclc.boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/train.boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/train.boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/boba.org-le-ssl.conf
Created an SSL vhost at /etc/apache2/sites-available/wp.boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-available/wp.boba.org-le-ssl.conf
Enabling available site: /etc/apache2/sites-available/wp.boba.org-le-ssl.conf
Deploying Certificate to VirtualHost /etc/apache2/sites-available/wp.boba.org-le-ssl.conf
Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration. 2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for new sites, or if you're confident your site works on HTTPS.
You can undo this change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Enhancement redirect was already set.
Enhancement redirect was already set.
Enhancement redirect was already set.
Enhancement redirect was already set.
Enhancement redirect was already set.
Enhancement redirect was already set.
Enhancement redirect was already set.
Redirecting vhost in /etc/apache2/sites-enabled/wp.boba.org.conf to ssl vhost in /etc/apache2/sites-available/wp.boba.org-le-ssl.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Your existing certificate has been successfully renewed, and the new certificate has been installed.
The new certificate covers the following domains: https://alanboba.net,https://boba.org, https://sclc.boba.org, https://train.boba.org,https://training.boba.org, https://www.alanboba.net, https://www.boba.org,https://wp.boba.org, and https://www.wp.boba.org

regex – headaches and solutions

Regex could save your bacon. Try it.

Need to parse a string that doesn’t have a consistent layout but does contain regular patterns that need to be keyed on to extract information (or change it)? Want to extract values from a string without complex nested IN(…, MID(…, SUBSTR(…, LEN(… functions?

Regex seems to be the tool to use, if you can wrap your mind around the syntax for patterns that need to be keyed on. There’s plenty of examples to be found that illustrate searching for phone numbers, social security numbers, ZIP codes, ZIP+4, IPv4 addresses and so on. Add in features like look ahead, look behind, greedy or non greedy searching and I get the feeling there’s more than enough material to develop an entire graduate level course to formulate and understand regex expressions.

My own forays into regular expressions have focused mostly on mundane simple patterns. I did however find a couple uses for the look behind to extract a substring from an irregular length string. And the substring to extract was not fixed length. To me the function based way of parsing the string would have been much more complex than working out the regex syntax.

In one case there was a substring for the size of a hard drive. That substring varied in length depending on the size of the drive. In another case strings for monitor make and model, which could each be different lengths because more than one manufacturer was in the dataset and model names varied.

In both cases the target string was preceded by a fixed string that could be used to locate the sought for value within the larger string. And the position of the target string within the larger string varied because the details before it changed from pc to pc. Rescued by the regex look behind!

I posted about it on LinkedIn if you’re interested to look. Samanage and Powershell – two tools to produce helpful reports.

The significant portions of the statements are below. They allowed looking into the string for a pattern, `Fixed hard disk media”~size: “` or `name: “` or `manufacturer: “`, and then extracting the portion of the string that immediately followed.

$regexdrv = '(?<=Fixed hard disk media"~size: ")([^"]+)'
Select-String -Pattern $regexdrv | %{$_.Matches}
$regexdisp = '(?<=name: ")([^"]+)|(?<=manufacturer: ")([^"]+)'
Select-String -Pattern $regexdisp -AllMatches | %{$_.Matches}

If you find yourself struggling to parse out some information from a string, take a look at regex. You could find the solution to your problem.

Remote Support

Using RealVNC and AeroAdmin to help people at their computer

I provide remote computer support to people. Mostly to family but also occasionally to friends and acquaintances.

For family I’ve been using RealVNC. For other folks, AeroAdmin.

RealVNC has several advantages from my perspective. Two important ones are there’s a native Linux version. Family doesn’t use Linux but I do and I want to be able to run something directly on my system rather than running it through layers. Secondly and most importantly for family, it’s running all the time on their system. If support is needed we get on the phone, I start the connection, a message pops up on their screen and they click OK to allow the connection. Shazam! remote support has started. There’s nothing for them to install, no program for them to start. Just ask for support then click OK.

Other pluses are it runs on Linux so I have remote access to my own system when I’m away from it and I can register up to five systems in my RealVNC account with no subscription license required.

For the person I’m tutoring now and others I’ve helped in the past AeroAdmin has been the way to go. There’s a free license that allows occasional usage without charge. When I say “occasional” I don’t mean to imply crazy limited. From my perspective the free license is very generous with the time and connections allowed. I have certainly never bumped into the usage limits.

Of course there’s drawbacks to both. With RealVNC the issue is running it on my system. RealVNC occasionally freezes and restarts. When it does I either have to wait out the freeze before I can begin using my pc again (same experience whether I’m connected remotely or at the keyboard) or I restart my system if I’m at the keyboard. An issue I choose to live with because it gives me remote access to my system and I know how to deal with it. When I’m at the keyboard I sometimes just shut down RealVNC so there’s no chance it interrupts me. I really should file a bug report but haven’t bothered yet. And since it’s me that’s affected, not family I provide remote support to, I know what’s going on and corrective actions I can take.

EDIT 3/2020: I found the issue back in January. A wall paper changer set to change wall paper every five minutes. Increased the interval to 15 minutes and the problem became less frequent. Switched to a different wallpaper changer and the problem has gone away

AeroAdmin is a different story. The biggest challenge is always getting the person who needs to use it to get it on their system, get it started, provide me the information I need to connect, and finally make the correct choices when the connection starts so I can actually help. That sounds hard but it’s not. But it is for many people.

I’ve made documentation and sent it to people. I’ve recently made this video, AeroAdmin, letting someone help you. But it is still a challenge. Getting the connection started so people can get help always seems to be the biggest issue.

Another AeroAdmin drawback, for me not the person supported, is that I need to run Wine to run AeroAdmin. I’ve never been a fan of Wine because whenever I tried it my system was changed in ways that I couldn’t easily undo. Plus, depending on the install package, it added things to my system that I didn’t want. Getting stuck with things I didn’t want was one of the reasons for getting away from Windows. However AeroAdmin does run well under Wine so, when I need it, I run it in an isolated Linux vm with Wine installed so Wine isn’t impacting my actual running system.

Certbot automatic authentication

Enable certificate auto renew after a manual renew.

I have a number of websites run from my own web server, like this one. Something I set up to experiment with web technologies and gain some insight into how things work.

One of the things I did was setup HTTPS for the websites once I found about about EFF‘s LetsEncrypt service. I wanted to see if I could provide secure connections to my sites even if they’re only for browsing.

I was able to get HTTPS working for my sites and have the certificates renew automatically. Then I changed ISPs. With TWC, now Spectrum, there was never a problem with the automated renewals. With Optimum the renewals didn’t work.

Emails alerting me to certificate expiration were my first indication there was a problem.

The logs indicated that files on my server couldn’t be manipulated to confirm my control of the website. Plus, entering the website address as boba.org or http://boba.org no longer connected to the website (externally, on the local network it still worked). Connection to any of my hosted sites now required prefixing https:// to the name. Automatic translation from http to https no longer worked.

After talking, chatting online actually, with Optimum they told me yup, that’s just the way it works. “We block port 80 to protect you” and “you can’t unblock it”.

Panic! How to maintain my certificates so https continues working? Fortunately certbot offers a manual option that requires updating DNS TXT records. It’s slow and cumbersome and NOT suitable for long term maintenance of even one certificate containing one domain but it works.

Sixty days pass and the certificate expiration emails start again. This time I determined that I’d speak to a person at Optimum and not use the chat. After some time with my Optimum support tech, and after she escalated to a supervisor, I was told there is in fact a way to open port 80. And it is a setting available to me via my account login. So I opened port 80 and thought all set now, renewals will happen automatically.

Not so. I got more certificate expiration warning emails. What to do? All the automated renewal tests I tried indicated a problem with a plugin. I read the certbot documentation, did searches for the error and tried to find a solution that was applied to the problem I had. I didn’t find it. But I did get a clue from a post that said once a manual certification has been done that setting needs to be removed before automated renewal will work again.

After more digging I discovered the certificate config files in /etc/letsencrypt/renewal. In them were two variables that seemed likely to be related to the auto renew problem. They were authenticator = and pref_challs =. The settings were manual and dns-01 respectively.

I never touched these files. It turns out doing manual renewal with DNS TXT records using the command sudo certbot certonly --manual --preferred-challenges dns --cert-name <name> -d <name1>,<name2>,etc just changes the config files in the background. Attempting auto renew later doesn’t work because the settings in the config files have now been changed to authenticator = manual and pref_challs = dns-01.

There was no help I could find that explicitly listed the acceptable values for these variables. And I didn’t have copies of these files from before the changes. After digging around in the help for a while I decided it was likely they should be authenticator = apache and pref_challs = http-01.

I made the change for one certificate and tested auto renew. Eureka, it worked!!

Next I changed the config files for all the certificates and did a test to see if it worked.

$ sudo certbot renew --dry-run ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates below have not been saved.) Congratulations, all renewals succeeded. The following certs have been renewed: /etc/letsencrypt/live/alanboba.net/fullchain.pem (success) /etc/letsencrypt/live/andrewboba.org/fullchain.pem (success) /etc/letsencrypt/live/danielboba.org/fullchain.pem (success) /etc/letsencrypt/live/kevinkellypouredfoundations.com/fullchain.pem (success) /etc/letsencrypt/live/www.anhnguyen.org/fullchain.pem (success) /etc/letsencrypt/live/www.conorboba.org/fullchain.pem (success) /etc/letsencrypt/live/www.mainguyen.org/fullchain.pem (success) ** DRY RUN: simulating 'certbot renew' close to cert expiry ** (The test certificates above have not been saved.)

It worked. All my certificates will again auto renew.

This website was created after the problems began. So I didn’t even attempt to make it https. Now that I’ve figured out how to have my certs auto renew again I’ll be converting this site over to https too.

Windows 10 images

Windows’ various versions are the computer operating system I’ve supported my entire professional career. There have been very occasional instances of supporting other systems like Mac’s OS, both before and after Apple switched their OS to UNIX.

There’s many things I don’t like about Windows. I’ve stopped using it for my personal systems for around a decade now. One of many gripes is the installation and update process.

For a while I was fortunate enough to have a professional staff who developed Windows deployment images for our company. They were very good and made image deployment “just work”. It was to the point that about all that was necessary was network boot the pc, point it to the image source and sit back and wait.

I reviewed the procedures they created. Asked questions to better understand what needed to be done to create the Windows images. I never actually was hands on creating an image though. Not from my staff’s documentation and not with any of them shoulder surfing me through the process.

Years later I reached the point of needing to create zero touch deployment images on my own. I failed. It seemed I was close to the solution but never quite there.

Microsoft’s documentation is terribly frustrating for me for the task of image creation. I’ve not found a single Microsoft webpage that goes from zero to bootable deployment image. There’s lots and lots of webpages with instructions for various portions of the work. And some webpages with basic outlines that have links (too many) to details that themselves have many links to more details. Alice never went down such a deep rabbit hole.

Then I found Kari Finn`s guide to “Create media for automated unattended install of Windows 10” on tenforums.com. Kari takes all the diversions Microsoft provides and narrows them down into a single linear process that goes from having installation media to having a zero touch custom installation image. BRAVO and thank you Kari!

Using the guide I’ve finally made my first successful zero touch deployment image!!!

From here I’ll make custom images for the software installations and architectures, BIOS/MBR and UEFI/GPT, that I need to support.

Finally I can make my own images. The world is my oyster.