Introduction

So far, at the University of Iowa, I’ve taught two classes on Python (as of this writing), and now at UFG, the trend I’ve noticed for awhile now is worth talking about. In my class, and at my workplace, programming seems to be assumed best done by programmers and because it’s “someone else’s’ job, why should a non-developer learn it?”

Before we go onto examples, let’s define what I mean by programming because it’s likely more broad than most people would expect.

Programming is the process of automating a process done on a computer to reach some end goal.

It’s as simple as that. From web development, to building Machine Learning models, we’re building a process that can make us (either individually, or as an organization) more efficient. Let’s take some traditional examples of Programming, and see how this definition applies.

Web Development (building a web page)

Many websites are store-fronts. You visit a site, you buy something, they ship it. Think back when QVC used to actually be a useful thing. How did you buy stuff from QVC? You called them. Also, since it was on T.V., you had live people demonstrating said items, and you’d have to physically be watching and physically call in. Programming, in the end, was a desire to replace a store-front and clerks with an automated process. Thus, making the business more efficient.

Server Maintenance (scripting)

Scripting is a form of programming. Maybe not as complicated at times as full-blown projects, but it’s still programming. Instead of having a person physically create a VM, load the Operating System, and configure it, often times you now have a script that does all this.

Why learn programming, what’s in it for you?

It’s my honest opinion that everyone can, and should, learn a programming language. One key word there is ‘a’, not multiple. You may need to use more than one programming language, but you should know one. The distinction is important. When you know something, you understand the concepts behind that something. Since the majority of programming languages are similar, or at least fall into a few categories, using an unfamiliar programming language shouldn’t be that difficult once you know one language.

So this goes back to the question, why learn a programming language?

In both of my classes at the university, I’ve primarily had students who have never used a programming language before the course. They came from a Business background, and were interested but didn’t understand how they’d use it. One thing I did during class was create supplemental recorded lectures over specific use cases for the language. Things that apply to the real world. Some of these included:

1. The ability to read from a SQL database - Let’s assume that you work in finance, and need to pull some information to investigate something you see in Payroll. If the company allows for access, and has well defined schema and Data Governance policy, you could pull this information yourself to get it into a format that you need. This can help with other teams when providing requirement documentation.
2. The ability to read from, combine, slice, Excel files - Excel is everywhere, and often times the output of various reports is in Excel. If you’re pulling reports from multiple sources, exporting them into Excel, you can use Python and Pandas to read from these, and join them in various ways. You can do some of this in excel (vlookups, PowerPivot, etc.) but the power of using Python offers greater flexibility in how that’s done.
3. Automate Parts of your Workday - Let’s assume you have to email reports out to people on a regular basis. You could write a script to handle the majority of that operation. Having it send an email, as yourself, to those individuals who need it. Depending on the volume of these, this can scale.

At UFG, there were a few specific business cases that came up recently, that I’m helping with:

1. Automate ETL Processes - We use a project called Stonebranch, but given the architecture of it, we wanted to avoid duplication of jobs, provide something that scales, and so on. I wrote a Python script that has a lot of flexibility for workflow-based execution of the ETL processes within the script. That way you can have the same operation within Stonebranch (with variables pointing to the right server), and the configuration is handled in a config file the Python script will load.
2. Converting a very large PDF into an Excel file - The reporting team was trying to determine some problem data in a report. This report is ~150 pages. Converting to Excel (from Adobe) is not possible, nor is Word or Text. I’m still working on this, but the idea is to have a small Python process that can read the exported text file and create a Pandas DataFrame which can then be exported to Excel.

Even with my own job, there’s a good deal of Python. Which, one wouldn’t expect given how I’m architecture systems:

1. Data Governance - I’ve been working on a process to pull in the Data Lineage between different systems. A lot of our report models are in XML, as is an export of our ETL process. There’s SQL Servers also involved, and data specifications in Excel. Either way, to get this all into a unified format is challenging, and is why I’m using Python to assist.
2. Data Verification - Another thing I’ve been working on here and there is the verification of data between two systems. This, also, is using Python to load data from both, do checks, and present a report. Along with that, I also have notebooks that help me find data I need by column, table, etc.

So, in short, I think some Programming is useful for everyone to know.

How to get started, what to pick?

What you start with is entirely up to you, but I do think a language that has a core conceptual basis is important to learn. It may seem pointless, but the concepts of a language will translate to other languages you have to use. So it’s one reason why I don’t suggest something like PowerShell, Bash, or the like to start with. For most Business uses, I promote Python more than anything else. Why?

1. Python is easy to learn, relative to other languages (such as Go, C#, Java, etc.)
2. It has a structure associated with it. Love it or hate it, but the tabs and the syntax structure are good for newcomers.
3. There are many, many libraries out there. And, for many of the Business cases one runs into, Python should help a lot.

Does that mean you shouldn’t learn PowerShell, Bash, etc? Of course not. I just feel that it’s good to have a good solid conceptual basis that you can build off of. You can also really pick any language that has concepts of Types, iteration, objects, and so on. Which, yes, I know Powershell has, but I don’t think it’s conducive for learning the concepts well. I’d also warn against Perl for a similar reason.

So once you pick your language of choice, how do you get started?

If you’ve picked Python, then I recommend looking into PyCharm. Visual Studio Code is also good, but requires some extra setup. I don’t recommend using normal Jupyter Notebooks to learn off of, despite this is what the Business class used. The students who ended up using PyCharm did considerably better on the conceptual aspects than those who didn’t. Largely this has to do with the ability to debug their application(s). I also recommend The Quick Python Book - 3rd Edition. But, Python education is everywhere - if you have Pluralsight, O’Reilly, or Linkedin-Learning, check there too.

If you’ve picked something else, then the answer is more difficult, but similar to that if you picked Python. Visual Studio Code is worth looking at. When in VSCode, you can view the ‘Plugins’ section, which will allow you to search for the language you’re using. Of course, you’ll need to install the language runtime and packages, as well. For resources, I can’t emphasize enough how good O’Reilly and Pluralsight are for these. They cost a fair amount of money, but are fantastic resources for learning.

Learning your programming language

The one last thing I wanted to mention on this topic is how to effectively learn the programming language you decided upon. Another thing I noticed from the students in my course is, unless they had an ‘end goal’ in mind, they had a hard time working with the concepts in the course. So, the most important part of all this is asking yourself the following:

What do I do, that’s repeatable, that I’d like to remove from my concern in a day.

By repeatable, think of anything. Also, ensure you consider things outside your work as well. Do you do anything repeatable on your computer at home that you can automate or “do better”? Having a clear goal can easily help in overcoming the inevitable frustration that you’ll likely feel while working through all this. Another thing I noticed with students is that a little bit of work, every day, is much better than bunching the learning to the weekend. 15-30 minutes, even if it’s a review of concepts you’ve already worked on, will help you out a lot. This is something an old coworker taught me.

Expect to be frustrated trying to learn your first language. Expect it to feel like it’s taking too long. Also note that you’ll likely develop solutions much slower than you could have done something manually when you get started. But, the overall goal is that over time the investment you are making will pay off later. Just like anything in life, effort is required and delayed gratification can help you a lot.