Understanding User Requests

September 11th, 2018

bobwaycott at ycombinator made a comment that summarizes my thoughts on developing software for end users.

Always. I’ve been practicing this for 10 years, most of which I’ve been working as a consultant building custom software for internal business processes, as well as customer-facing software for clients’ users. My typical process looks something like this when a feature/change request or random idea is asked for—which almost always comes in along the lines of, “Can we get X added in that does Y and looks like Z?”:

1. I never say yes to any request immediately. I always tell a client, “Anything is possible that doesn’t violate the laws of physics, but let’s dig into this more.”

2. Ask what they’re trying to accomplish. What problem are they trying to solve? What mistake are they trying to prevent? What is the end goal? I ask questions until I can explain back to the customer what they’re really wanting to do and why.

3. I the push back on why I think we should not do what they’re asking for in the way they’re asking for it. When doing this, I always try educating them on the tech behind the scenes, potential pitfalls, how adding something (especially if it’s visible to people) will make it nearly impossible to ever remove it once users get used to it, when they’re asking for something that would be more effort the way they’re asking for it to be done than it’s worth when it’s trying to solve for a rare edge case, etc.
4. After explaining the problems with their idea as given by non-experts, I start suggesting ways we could accomplish their goals with a simpler UX, or even no UX at all, relying on the ability to automate things if we have enough information, or hide all the complexities of a process behind a single button once we have the right info to intelligently take action.

5. I give a couple of recommendations for potential ways forward that solve the real problem in a way I’ll enjoy building it out. Then I let them make the choice.

Following this pattern has pretty much never failed me. What feels best about it is when I see clients actually learn how their software works when I’m working for them. I love it when they remember the discussions we’ve had, internalized it, and recall it when we talk 6 months later about their new idea. Over time, their ideas improve because their understanding of how their software works improves. They also become increasingly invested in our working relationship as their trust in my concern for solving their problems—and not just doing their bidding—increases.

Never shy away from challenging your customers’ ideas—but always do it in a respectful manner that gets to the heart of their real problems and educated them along the way. They’ll appreciate it, and will keep coming to you for more. I don’t think this is unique to being a consultant, either—the same sort of process can be followed with direct users of your own product

Updating Ubuntu LTS

August 22nd, 2018

I have a small site that was running Ubuntu 16.04 LTS and upgraded it to 18.04 with no issues. I have a couple of customizations in my php.ini and the location where I store my email and because I accepted the maintainer’s version of the PHP and Exim4 files, I had to update a few files to get everything working as before. Not a big deal and the whole process took about a half hour.

On my main site, I was running Ubuntu 14.04 LTS and it was a major project to update. There were times that I considered reverting to the snapshot I took just before I started the update process, but I got everything working about the same as it was before.

The first issue I encountered was with the update to the Linux kernal. The instructions recommended that I go to the Linode control panel and change the Boot Settings Kernel to Grub. When I did that, the server wouldn’t reboot. So I picked the Latest 64 bit version instead and rebooted fine.

Because I was on an older version I had to update in stages. Somewhere along the line it messed up my MySQL install. So I removed MySQL and installed MariaDB. That ended up causing a lot of work which I’ll get to later.

Restoring mail service was pretty straightforward—just changing the location of the virtuals files.

After working through the install process I loaded all of my sites and some displayed properly, some displayed without most of the formatting and some displayed a blank page. It took a while to figure out what was going on. WordPress sites weren’t displaying because of database access issues. There was lots of really weird behaviour on the other sites that I eventually tracked down to permissions problems on a shared styles folder. For example, if I accessed a site with sitename.com things looked fine. But if I accessed it with www.sitename.com, I got no formatting. And in one case it redirected to the default site for the server. It took quite a while to figure out what was happening but eventually I tracked it down to a permissions problem. I keep styles that I use on all my sites in a sim-linked folder in /www/common/Styles. After I changed all of the file permissions to -rw-rw-r– and the directory permissions to dr-xr-xr-x, the sites started behaving normally.

The MySQL issues were much more difficult to resolve and took the better part of two days. There were enough changes to the users file that I couldn’t just copy it into /var/lib/mysql. I went with the default install and then took then loaded users (and their database permissions) from my mysqldump from the night before. I took the opportunity to delete users who no longer needed access to the system and clean up some of the permissions on those who remained.

Copying the sql files from a backup copy worked for most tables, however there were some that only had a .frm file and no MYD and MYI file. PhpMyAdmin showed them as being “in use” and would not display their contents. I reloaded their contents from the mysqldump. That solved most of my website display problems, but I couldn’t log into any WordPress installations. That turned out to be a permissions problem. I changed the user and group of all the sql files to mysql and then changed all of the permissions on the files to -rw-rw-r– and folders to drwxr-xr-x. Now all of my sites were displaying properly.

The only problem was with the order confirmation page. I got an error Cannot start session when headers already sent error. I had a redirect on one page that was in fact trying to redirect after some html had already been sent. Not sure why it worked before, but putting in a conditional to not load the html if the page was going to be redirected fixed it.

The order page still didn’t work and the reason was that it used mc_encrypt which is no longer supported in PHP 7.2. I updated the files to use openssl_encrypt and everything works as before. I didn’t write the original order form code so it took a while to understand how the encryption works. There are lots of examples out there, including on the PHP.net site, so I won’t go into detail about what I did. Understanding and implementing the new encryption method probably took around three hours.

An older set of pages that used mysql_real_escape_string also failed to load because mysql_real_escape_string was no longer supported. In this case, replacing it with htmlspecialchars solved the problem and should have been what I used to begin with. This section of the site also had some date returns that did not correctly use the date function. I replaced $thisMonth = date(F); with $thisMonth = date(‘F’); and did the dame thing with $thisDay = date(d);

For some reason, aptitude wasn’t upgraded so I had to install it when I did the first update of the site.

Server Unavailable error

August 8th, 2018

A PHP-based web site on a server that I manage was failing to load. It worked a few months ago when I set it up and the owner didn’t notice that it was failing. When I looked at the error log I got these messages.

[Wed Aug 08 07:45:21.627045 2018] [proxy:error] [pid 32113:tid 139735343294208] (2)No such file or directory: AH02454: FCGI: attempt to connect to Unix domain socket /run/php/php7.2-fpm.sock (*) failed
[Wed Aug 08 07:45:21.627334 2018] [proxy_fcgi:error] [pid 32113:tid 139735343294208] [client] AH01079: failed to make connection to backend: httpd-UDS

Since it said that fpm.sock failed, I thought that maybe the mod was no longer enabled so I tried a2enmod proxy_fcgi but it said that it was already enabled.

There was some info on dependencies on various help sites, so I figured that at some point an aptitude safe-upgrade command might have messed things up. So I figured that running these commands might help, and it did.

sudo apt-get update
sudo apt-get upgrade

Reading through the install messages I noticed this:

Setting up php7.2-fpm (7.2.7-0ubuntu0.18.04.2) ...
Package apache2 is not configured yet. Will defer actions by package php7.2-fpm.
Replacing config file /etc/php/7.2/fpm/php.ini with new version
NOTICE: Not enabling PHP 7.2 FPM by default.
NOTICE: To enable PHP 7.2 FPM in Apache2 do:
NOTICE: a2enmod proxy_fcgi setenvif
NOTICE: a2enconf php7.2-fpm

So it looks like proxy_fcgi somehow got enabled and was causing problems. It is a small site with little traffic, so I decided to leave well enough alone and not enable the mod.

Banner day for scamming.

August 3rd, 2018

Two scams that are in the news hit our household today.

The first was a sextortion email. This one was a copycat though, since it had an an email address that was obviously found through a hack (vendorname@wellgolly.com), but they didn’t have the password to prove that they’d been monitoring the webcam.

And then I got to talk to a “Microsoft Security Team” member that was informing me that the internet on my computer was illegal. I tried to get him to clarify what he meant by expressing confusion about how the internet was on my computer, since as far as I know, it is outside my house, but I wasn’t able to string him along. And just for a point of reference, we only use Macs, iPads, and iPhones here, so there was no way that they detected “Illegal internet activity on my Windows”.

Getting Exim4 running on a new server

July 6th, 2018

We’ve always kept our virtual domains in /etc/mail/virtuals and I was under the impression it was the default location. It isn’t.

I don’t remember adding this code when I set up Exim4 on my Linode server, but it is missing on my Digital Ocean server. I needed to add the folowing lines to exim4.conf.template, just above the line system_aliases: in the router section.

  driver = redirect
  domains = dsearch;/etc/mail/virtuals
  data = ${lookup{$local_part}wildlsearch{/etc/mail/virtuals/$domain}}
  file_transport = address_file

I also missed the step of creating the self-signed key and cert, so make sure you do that as well.

The documents have lots of special cases for handling email that I never used, but one could come in handy. An attempt to deliver to a particular local part can be deferred or forced to fail by aliasing the local part to

So you could do something like this, since spammers use this address all the time.
support: :fail:

One thing we never did in the virtuals file is to send a comment for addresses that bounce, but it could come in handy. e.g

X.Employee: :fail: Gone away, no forwarding address
support: :fail: Please use the contact form on our website if you have support questions.

Well Golly

Atheism Plus

Buy from Amazon