Duo Security Lab – Multi-factor Authentication for SSH

Part of a series of posts related to the cloud security company Duo Security, Inc. I am not affiliated in any way with Duo Security (please read my more extensive disclaimer below), I’m just doing my best to understand their offering.

Following the very clear instructions I found on https://duo.com/docs/duounix, I proceed to Duo my SSH. This is my topology:

I start by getting my Ubuntu 18.04 Server ready. Duo says we’ll be building from source and I need a compiler like GCC so I just installed the build-essential package. I also need libssl-dev and libpam-dev.

sudo apt-get install build-essential libssl-dev libpam-dev

Then I need to get the pam_duo source, untar and compile.

wget https://dl.duosecurity.com/duo_unix-latest.tar.gz
tar zxf duo_unix-latest.tar.gz
cd duo_unix-1.11.2
./configure --with-pam --prefix=/usr && make && sudo make install

The make program then spits out a bunch of gibberish that I will never understand, but I know it compiled ok because it didn’t say “error” a bunch of times. I’m very precise like that.

Next up I copy the keys and API hostname (the magic link to my Duo account) found when I click “Protect this application” on my Duo account under “applications”:

Then I put them in /etc/duo/pam_duo.conf file. It should look like this, according to the Duo documentation:

[duo]
; Duo integration key
ikey = DIGHFMF9NB25BEQJGHQZ
; Duo secret key
skey = NOTGONNATELLU
; Duo API hostname
host = api-b19c01d1.duosecurity.com

Then edit the pam sshd configuration file /etc/pam.d/sshd, find this line and comment it:

@include common-auth

Then add three lines so it looks like this:

auth  [success=1 default=ignore] /lib64/security/pam_duo.so
auth  requisite pam_deny.so
auth  required pam_permit.so

Lastly, restart the sshd daemon to apply the changes to sshd and duo.

sudo systemctl restart sshd

Once I get that all in place, I try SSHing (is that a word?) from the client to the server:

james@client:~$ ssh james@142.69.3.42
Please enroll at https://api-b19c01d1.duosecurity.com/portal?code=ca487343d044ab35&akey=DAVEM6A3YS7ML96IE4BJ
james@142.69.3.42's password:

I need to enroll my MFA device (my android phone) at https://api-b19c01d1.duosecurity.com/portal?code=ca487343d044ab35&akey=DAVEM6A3YS7ML96IE4BJ, so I head on over there and follow the prompts:

Then I try again SSHing from the client to the server. This time I get an MFA prompt on the command line:

james@client:~$ ssh james@142.69.3.42
Duo two-factor login for james

Enter a passcode or select one of the following options:

 1. Duo Push to XXX-XXX-7273
 2. Phone call to XXX-XXX-7273
 3. SMS passcodes to XXX-XXX-7273

Passcode or option (1-3): 1

Entering option 1 gets me a prompt from my Duo Push app on my phone to accept or deny the request (the app restricts taking a screenshot so I couldn’t include it, sorry), options 2 and 3 are pretty much what they look like.

Success. Logging you in...
Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-20-generic x86_64)
Last login: Mon Jun 24 20:35:10 2019
james@server:~$

There is much rejoicing.

Non-Affiliation Disclaimer:
I am not affiliated, associated, authorized, endorsed by, or in any way officially connected with Duo Security, or any of its subsidiaries or its affiliates. The official Duo Security website can be found at https://duo.com. The name Duo Security as well as related names, marks, emblems and images are registered trademarks of its owners.

Duo Security – Overview and Target Market

Part of a series of posts related to the cloud security company Duo Security, Inc. I am not affiliated in any way with Duo Security (please read my more extensive disclaimer below), I’m just doing my best to understand their offering.

History and Products

Duo Security is a cyber-security company based out of Ann Arbor, Michigan, founded in 2009 by Dug Song and Jon Oberheide. In August of 2018 they were acquired by Cisco Systems. Duo’s LinkedIn profile makes a pretty clear and concise statement that they’re going to “democratize security” and that their mission is to “protect the mission of our customers by making security simple for everyone.”

Unaltered screenshot of Duo’s Product page as of 04/29/2019

Duo’s product page makes some pretty big claims about what they can do. Their product lineup targets securing apps and data, but what stood out to me is that they say it works from any location using any device for organizations of all sizes. Duo offers a platform called “Trusted Access” that has multiple parts:

  • Multi-Factor Authentication
  • Endpoint Visibility
  • Adaptive Authentication & Policy Enforcement
  • Remote Access & Single Sign-On

I’ll take a good look at what these actually mean for their customers later, but for now it’s clear they aim to secure and authenticate their customers’ systems.

Duo’s Customers – IT Departments Big and Small

It’s also fairly clear you probably wouldn’t deploy the Trusted Access platform’s features on your home WiFi network to enable trusted secure access to your Google Chromecast, as they target enterprises. They have a really nice use cases section on their homepage that shows some of the different verticals they’re after including:

  • Education
  • Federal
  • Healthcare
  • Legal
  • Retail
  • Technology
  • Finance

I took a look at one use case in particular for their customer Etsy, an online retailer of handmade or “vintage” items.

Authentication: not as easy as it looks. Photo by Jason Blackeye on Unsplash

According to the case study, Etsy’s business problem centered around securing administrators’ access to the internal management systems of their site. They use a number of access tools including SSH and internally developed systems.

Etsy cited “single-factor” authentication as a security problem for their organization, a.k.a. authentication with only a username and associated password between the outside world and access to said management systems. Duo quotes Etsy’s Network Security Manager describing Single-Factor Authentication as a “weak-link” to illustrate this issue.

Etsy used Duo’s Multi-Factor Authentication feature to add another factor to the authentication process for administrators accessing internal management systems of the site. There are multiple options for adding a second factor to the authentication process (which I’ll explore later), but Etsy says they used the Duo Mobile app. The app enables “pushing”, or the sending of an authentication request (after entering the correct password) from Duo’s Trusted Access platform to the app on the administrator’s phone. The administrator approves access from her phone, and is allowed in to the internal management system.

Next I’ll take a closer look at the different features the Trusted Access platform offers.

Non-Affiliation Disclaimer:
I am not affiliated, associated, authorized, endorsed by, or in any way officially connected with Duo Security, or any of its subsidiaries or its affiliates. The official Duo Security website can be found at https://duo.com. The name Duo Security as well as related names, marks, emblems and images are registered trademarks of its owners.

Python Primer 4 – Functions

This is my fourth post in a series to quickly introduce Python:

I find functions to be convenient and useful, but completely unnecessary to writing a basic program. You can write a really cool program in python that never uses a custom-written function because that’s what a program is, a function. Inevitably though, you’ll probably want to do one or both of the things that I find functions useful for – reusing code or scoping.

Re-using Code

If you write code for long enough, you’ll get to a point where you find yourself copying and pasting code. It’s okay, everybody does it at some point. But you’ll know you’re doing something wrong if it becomes a lot of work, after all that’s what you have a computer for.

Say for example you had a guessing game program that picked a random color out of the “colors” list that we’ve been using in the last few posts:

import random
colors = ["red","purple","blue"]
guess = 'red'
answer = colors[random.randint(0,len(colors)-1)]
if guess == answer:
    print("You guessed right! The answer was "+answer)
else:
    print("Try again! The answer was "+answer)

Line 1 is new. “import” means to pull code in from somewhere else, in this case from the “standard library” which is a bunch of code that the python creators built for you. When you import, you import a “module” which is basically just a file where the code in the module is stored. “random” is specifically a module that gives you more pre-built functions that do random things. You access these new functions using a dot “.” as in “random.randint()” which picks a random number in a range of numbers.

Let me also explain line 4 where the answer is created. Basically it boils down to this:

colors[] #selects one of the items from the colors list
random.randint() #creates a random integer in a range
len(colors)-1 #we get the length of colors minus 1 (starts at 0)

Putting those together ends up selecting a random item from “colors”. If the selected answer matches the guess, it prints “You guessed right!”, if not it prints “Try again!”).

If you write that program on the interactive shell and want to play the game more than once, you’ll be copying and pasting lines 3 – 8 as many times as you want to play. To avoid copying and pasting, we can put 3 – 8 in a function so we can reuse it easily. Functions are “defined” using “def”:

import random
colors = ["red","purple","blue"]
def guessing_game(guess=''):
    answer = colors[random.randint(0,len(colors)-1)]
    if guess == answer:
        print("You guessed right! The answer was "+answer)
    else:
        print("Try again! The answer was "+answer)

Now you can run your new function called “guessing_game” like so:

guessing_game(guess='blue')
Try again! The answer was red
guessing_game(guess='blue')
You guessed right! The answer was blue

The answer will be different every time because it’s selected randomly via the magic in the “random.randint()” function that’s part of the “random” module.

Scoping

The other reason to use a function doesn’t really make itself obvious until you’ve used functions for a little while. But I think it’s important to know so I’ll mention it here. The idea is that “inside” a function, you have a private space for variable names (a “namespace”) that does not affect the global space. Again, the use case for this is a bit hard to see when our program is so small, but I’ll show you how it works.

Let’s say we already have a variable named “answer” in our program before we created the function “guessing_game”. Maybe it was left over from some other function that ran, or your program is just really big. In any case, it’s there, just accept it. These things happen.

import random
answer = "magenta"
colors = ["red","purple","blue"]
def guessing_game(guess=''):
    answer = colors[random.randint(0,len(colors)-1)]
    if guess == answer:
        print("You guessed right! The answer was "+answer)
    else:
        print("Try again! The answer was "+answer)

When you run “guessing_game()” again, the “answer” variable that’s created inside the function every time you run it is unaffected by the global “answer” variable. That’s because “inside” the function is a tear in the space time continuum. I like to think of it like the “warp bubble” episode from Star Trek TNG. Everybody knows that one, right? Right?

guessing_game(guess="purple")
Try again! The answer was red
print(answer)
'magenta'

For my next and last post I’ll cover how to actually use python for something cool.

Python Primer 3 – Loops and Conditions

This is my third post in a series to quickly introduce Python:

This part is probably the most difficult to understand, but it’s really what makes programming dynamic and powerful. Loops and conditions logic (and most programming logic, actually) resembles the way people think and behave. I’d say a loop is kind of like eating, execute the same “put into mouth” action while in the “hungry” state. Conditions are pretty much the same as anytime you say “if” in your daily life – if snow is on the road, stay home, otherwise (else) go to work.

If Statement

To illustrate iffing in Python, I’m going to use two built-in functions, len() and print(). len() simply gets the length of something and “returns” it (gives it back) to you, in the form of an integer (a number). print() just writes what you give it to the screen. In the interactive shell, print is usually assumed when you press enter but it’s more useful in a script (more on that later). Using the same colors list from the previous post, I’ll get its length and compare it to 2 (because I can) using the greater-than sign “>”, perhaps you remember that one from grade school.

colors = ["red","purple","blue"]
because_i_can = 2
if len(colors) > because_i_can:
    print("I've got lots of colors")
I've got lots of colors

Notice how the print part is indented. Python requires this to let the interpreter know that particular code is “inside” the if statement. Don’t forget the colon after your condition, Python gets mad.

If you change “because_i_can” to 3, they’re now equal and the print code doesn’t run. This is the nature of iffing – only run the code inside if the condition is true.

There’s lots more to if statements, like “elif” which is just another condition you can add to an if statement, and “else” which runs if none of the if conditions ran. You can look those all up on the official documentation, and probably guess their syntax. But the simplest form only uses one “if”. I use it all the time.

Loops

There’s really just two basic kinds of loops, “for” and “while”. Their logic is designed to resemble the English words from which their names are derived. For – “for each something in a group of somethings, do something”. While – “while in something is true, do something. Stop when it’s no longer true”. I personally don’t use while very much although it certainly has its use cases. I mostly find myself trying to slap some data around to get it to do what I want, so I use “for” loops quite a bit. I’ll start with that.

colors = ["red","purple","blue"]
for color in colors:
    print("I like "+color)
I like red
I like purple
I like blue

“color” is a temporary variable that is created on-the-fly as you write the for loop, and “colors” is the group of stuff that you want to go through. The cool part is I only had write my print() code once, but it was executed for each color in colors. Let’s combine it with an if statement to make it a little more fun:

my_favorite_color = "purple"
for color in colors:
    if color == my_favorite_color:
        print("Purple is my FAVORITE color.")
    else:
        print("I like "+color)
I like red
Purple is my FAVORITE color.
I like blue

A couple of new things here – “==” is used to compare two things and see if they’re equal. I also added an “else” part to my if statement. It runs if the “if” part doesn’t run.

“While” statements are nice if you want to run some code while a certain something is true. I was quite satisfied with my food example above, so we’ll run with that. Let’s say our stomach can fit 3 potatoes in it. We’ll add potatoes to our stomach until there are 3 in there:

potatoes_in_stomach = 0
while potatoes_in_stomach < 3:
    potatoes_in_stomach += 1
potatoes_in_stomach
3

“+=” is a handy way to say “add 1 to this variable”. So 1 was added until potatoes_in_stomach was no longer less than 3.

Built-in functions are cool, but next I’ll take a look at how to make a custom function.

CPU Components – Adders

This post is part of a series in which I am following the structure of J. Clark Scott’s book But How Do It Know? (Affiliate Link – commission supports my knowledge quest). You can check out his website at http://www.buthowdoitknow.com/ (not an affiliate link). It’s written with no assumption of knowledge or background in computers, which has been great for me 😉

How Does an Adder Work?

As its name suggests, an adder takes two binary values and adds them together. My first thought was “that’s cool but how do you make it 32 or 64 bit?” which is a reasonable question. We’ll get to that, and it’s fairly straightforward. But understanding the simple (half) adder is key to understanding how the more complex ones work.

So if A = 1 and B = 1, this contraption using an XOR and AND gate will add them together to get O (output) = 0 and C (carry) = 1. Binary 10, aka 2. 1 + 1 = 2 right? You’ll find if you change out other values for A and B they’ll come out correct.

This also serves to illustrate that there are 10 kinds of people in the world, those who understand binary and those who don’t! Haha! ………ahem.

The Full Adder

A full adder is just a simple adder with another input. It’s called “carry in”, because the idea is to string together however many adders you want so you can add together 2, 4, 8, 32, 64 or some other number of bits. The full adder logical implementation is more complicated, but it works (I don’t show it here, I didn’t find it particularly important for the discussion. Also I’m a bit lazy).

So when you hook two together, the carry out of the first adder in the chain becomes the carry in of the second adder in the chain.

So if the the two A’s were the two bits of the first number you want to add, and the two B’s are the two bits of the second number that you wanted to add to the first, for example 10 and 11, the first O would be 1, the second would be 0, and the Co at the far right would be 1.

2 + 3 = 5 (binary 10 + 11 = 101), am I right?

J. Clark Scott (affiliate link, supports my knowledge quest) makes a good point – it’s an amazingly simple device for what it does. Although as it turns out, simply chaining adders together produces a fairly inefficient device for chemical and physical reasons. Modern adders are apparently more complex, but faster. It’s a pity life can’t be simple.

Python Primer 2 – Lists and Dictionaries

This is my second post in a series to quickly introduce Python:

Hopefully after the first post we’ve got an idea how to fire up the Python shell, create a variable (a cardboard box) that has some data type – an integer (a number), or a string (a letter or a number, a word, multiple words, multiple lines, whatever you want actually). A quick note about data types – the “type” determines how the data acts. An two integers can be added together like numbers, e.g. 1+1 = 2. But if you have a number inside a string data type, you can’t do mathy stuff. If you do “1”+”1″ you’ll get “11” because they’re strings, and that’s how strings behave.

As it turns out, you can bundle these data types together using things called Lists and Dictionaries. Don’t get discouraged if these make your head hurt at first, it hurt for me too when I first learned about them. And it may be hard to see how or why you might use these things at first, but rest assured they are used extensively.

The List

A list is pretty much what is sounds like. You can get your stuff in the list by referring to its order in the list, thus its called an “ordered” list. Lists start with “[” and end with “]” (called square brackets, or just brackets). List items are separated with commas. Typically you store your list inside a variable, like so:

colors = ["red","purple","blue"]

So if you want to grab just the first item in the list, type “colors”, immediately followed by the list item number enclosed in brackets:

colors[1]
'purple'

Was thinking I would get ‘red’, but actually everything in computers starts with 0. So colors[0] = ‘red’, colors[1] = ‘purple’, and colors[2] = ‘blue’ (Single and double quotes are interchangeable).

Python has a plethora built-in “functions” that I like to think of as mini-programs. You can get a whole list of them and much more at the official Python Documentation. I only mention this because we’re going to use a built-in function to do something fun. Fun if you’re a nerd, like me.

If you wanted a string like “red” to be upper case, but didn’t want to go to the work of redefining it, you could simply use a string function that already exists for it. Python says we need to use a dot “.” after the string we want to make uppercase. Also since it’s a function and “upper” is its name, put some parentheses after it to tell Python that you want to run (execute) it, because that’s how some people a million years ago decided execution should be notated:

"red".upper()
'RED'

But instead of re-writing the string, we’ll just grab it from the list instead since it’s already there:

colors[0].upper()
'RED'

The Dictionary

A dictionary is a way of doing a similar thing to a list, but the items aren’t in any particular order. You grab a particular piece of data (Python calls it a “value”) in the dictionary by referencing its “key”, which is just another piece of data. Usually people refer to dictionaries as being a database of key/value pairs, but that is really only useful if you already know how it works.

I’m sure this has been used in many other tutorials, but I think a good example might be of a set of stock market tickers and their associated share prices. The ticker is the key, the price is the value. A dictionary uses (curly) braces at the start and end, each key/value pair use a colon between them, and pairs are separated by commas. Again, it’s usually stored in a variable:

stocks = {'GOOGL':1189.84, 'AAPL':186.79, 'FB':167.68}

If you wanted to grab Google’s price, do the same dance as with a list, but use the key on the left to get the value on the right:

stocks['GOOGL']
1189.84

If you want to set a new value for ‘GOOGL’ since it’s price has changed, change it like you would a variable:

stocks['GOOGL'] = 1189.92
stocks
{'GOOGL': 1189.92, 'AAPL': 186.79, 'FB': 167.68}

One fun thing you can do using a built-in function is to add a new pair to the dictionary, by “updating” it (lists have a similar “append” function). For the list example I used a function on the string inside the list, here I’m using a function on the dictionary itself, hence “.update()” is attached to “stocks”:

stocks.update({'NFLX':359.97})
stocks
{'GOOGL': 1189.92, 'AAPL': 186.79, 'FB': 167.68, 'NFLX': 359.97}

I hope that’s useful. Next up is loops and conditions.

Python Primer 1

This is my first post in a series to quickly introduce Python:

I use Python a fair amount. Anytime I want to tell the computer to do something really specific that isn’t accomplished easily in some off-the-shelf software, Python is my go-to. There are blogs, books, websites, and others that can explain all the amazing things that this cool programming language can do, but since I foresee using Python to illustrate other concepts that I’m trying to learn and understand, I’m going to try explain what I feel are the most important parts in the fewest amount of words. So on that note, here we go:

Getting Started

Most places start you off with some fancy development software called IDE (Integrated Development Environment) but I don’t really like those and I think it’s important to try the basics first and work up from there.

Microsoft Windows

You’ll want to go to https://www.python.org/downloads/windows/ Just go through the installer and you should be up and running.

Mac (OSX)

For an Apple computer same deal, just go to https://www.python.org/downloads/windows/ and install. OSX comes with Python 2 but not 3, unfortunately, so it has to be added.

Linux

If you are using Linux, you probably don’t need to be told how to do this, and listing all the ways to install software on the various major distributions is a waste of cyberspace. I use Ubuntu, it’s already installed.

Fire Up The Shell

The “shell” or command-line is basically just normal Python in slow motion, because we humans type commands slow. Here you can type in commands to the Python “interpreter” one at a time (you can also paste them in for faster results). Python is “interpreted” as opposed to “compiled” (like C++ or Java) because it converts the code you write to machine code when you run your program, instead of doing the conversion beforehand as with compilation. This approach has advantages and disadvantages, places where it’s useful and where it’s not. But the command line feature is great for testing since you can write your code and execute it right there by pressing enter.

On Windows, just search for Python and click on what you find. You’ll be at the right spot if your terminal window looks like mine below. On Mac and Linux pull up a bash terminal type “python3” and you’re off to the races.

Python 3 on Windows

What can you do from here? For starters it’s a great calculator:

Or you can use it to scare your co-workers on Halloween:

But you’ll probably want to do something a little more interesting. For that we’ll need some of the basics.

The Variable

It actually took me quite a while to realize that a variable is just a box in which you put something. Behind the scenes a variable is just some space allocated in memory for whatever data you want to store, but Python handles all of that messy stuff for you since it’s a relatively “high-level” language, meaning a lot of dealing with computer inner-workings like memory addressing and CPU instructions is handled for you so you can focus on getting the computer to do what you want. Not that you should avoid learning about that, but in a lot of situations this high-level approach lets you get things done a lot faster. In any case, just think of a variable as a cardboard box. You can put numbers (Python calls them integers) in a variable. Press enter after you’re done with a line:

x = 1
y = 2
z = 3

x, y, and z are just some variable names I picked that have no meaning, really. Use the “=” sign to put a value in your variable. Now you can reference the names in your code, and not the numbers:

x + y + z
6

You can re-purpose your variables by reassigning values to them:

x = "The quick brown fox jumps over "
y = "the lazy dog."
x + y
'The quick brown fox jumps over the lazy dog.'

The part between quotes “” is called a “string”, meaning alpha-numeric characters strung together. A string can be just one letter, one word, multiple words, multiple lines, it doesn’t really matter (I’ll quickly mention that strings can use single quotes or double quotes, doesn’t matter which).

But don’t try to add two variables that aren’t the same “data type”, like an integer and a string, because Python gets mad:

y + z
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: must be str, not int

Welcome to the world of errors, you’ll be seeing a lot of those.

Next up is some cool ways to organize your stuff, Python calls them Lists and Dictionaries. See you next time!

Understanding Logic Gates

This post is part of a series in which I am following the structure of J. Clark Scott’s book But How Do It Know? (Affiliate Link – commission supports my knowledge quest). You can check out his website at http://www.buthowdoitknow.com/ (not an affiliate link). It’s written with no assumption of knowledge or background in computers, which has been great for me 😉

What Do Logic Gates Actually Look Like?

A computer is essentially a rat’s nest of fairly straightforward contraptions called logic gates. The two components that make a computer a computer are full of them – the CPU and RAM (See link for my post on how they communicate). They operate using electrical wires (turning them on and off to represent 1’s and 0’s) that serve as “input” (going into the gate) and “output” (coming out of the gate). There are lots of circuit diagrams out there that use symbols, but it’s kind of tough to get a look at what a logic gate might physically look like. The best photo I could find of a real device was actually here on Wikipedia (where all good things come from) on the page for NAND gate. I like it because it juxtaposes the symbol diagram alongside the actual chip. (There was no attribution information for this photo, please let me know if you find it. I’d be more than glad to give credit where it’s due)

YouTuber LPG‘s “redstone” computer.

J. Clark Scott’s book purposefully avoids talking about the physical construction of such devices, as it’s outside of the scope of the discussion of logic. Since you can’t really see anything that actually shows the internal workings of the 7400 chip shown above, perhaps the next best thing is a really amazing YouTube video on a computer constructed inside Minecraft (a computer game focused on building stuff with blocks, in case you’re not familiar). My hat’s off to the creator, I’m in awe of his creation. In any case, suffice it to say that logic gates are constructed by some physical medium using using chemistry and physics that I don’t yet understand. For now, at least, that awesome YouTube video will have to do.

Universal Gates

What struck me in learning about logic gates was that a number of gates are actually just combinations of other gates. There seems to be a bit of confusion out there on various sites regarding what gates are used to construct what gates, but the general consensus seems to be that NAND (negative-AND) and NOR (negative-OR) are the universal building blocks to build other gates, especially ones that are a little easier to understand like AND, OR, and NOT. In any case, most places purport the existence 7 gates (AND, OR, NOT, NAND, NOR, XOR, XNOR), but the best site I found was an article at All About Circuits on logic gates that shows a total of 16 (although some of them aren’t really gates or binary).

The reason why NAND and NOR are used to build everything else seems to be that the are easily built physically. If I understood more about chemistry and physics I could probably give you a more specific reason based on physical and chemical properties, but most descriptions I can find just say that you need to trust that NAND and NOR are built, then the rest are assembled with multiple interconnected NAND and NOR gates. For example, I have an AND implementation using two NAND gates shown below:

This contraption will turn O on only if both A and B are turned on, just like an AND gate.

How Does It All Fit Together?

Another shameless plug for J. Clark Scott’s book But How Do It Know? (Affiliate Link – commission supports my knowledge quest) because he does a really good job explaining how you would put these gates together to store 8 bits, which is also called a byte. Typically the register size is the size of the CPU’s computational “width”. Nowadays most CPU’s 64 bit, so their registers would be 64 bit as well, although in some cases there may be smaller ones. This tutorial on howstuffworks.com also does a good job explaining registers (didn’t like the section on the gates themselves though, heads up), and uses some terminology that you’ll definitely come across looking at logic gates, such as flip-flop and feedback. Storing a collection of on or off bits into a collection of fancily interconnected logic gates using a “set” wire as input allows you to store the state of the bits when the set wire was activated. Another wire, called “enable” allows you to access or read the bits. Such a collection is called a register, allows you to write bytes and read bytes. Amazing.

You’ll see registers all over the place, usually with a word or words before it to specify what its purpose is, such as Memory Address Register that says you want to access a certain address in memory and transfer the contents stored there (also in a register, called a Memory Data Register) to the CPU, where it will likely store said contents temporarily in a CPU register. All of the actions I have described are executed using combinations these gates, using voltage rising and falling as signalling to represent 1’s and 0’s across the wires. Some of the circuit diagrams and logical collections of gates in modern boards, memory and CPU’s can be intimidating, but I just try remember back to how simple the gates themselves are, and it makes me feel better.

3 Ways the CPU and RAM Communicate

This post is part of a series in which I am following the structure of J. Clark Scott’s book But How Do It Know? (Affiliate Link – commission supports my knowledge quest). You can check out his website at http://www.buthowdoitknow.com/ (not an affiliate link). It’s written with no assumption of knowledge or background, which has been great for me 😉

CPU/RAM Communication – An Elevator Pitch

The CPU and RAM of a computer communicate using system of wires, called a bus. This bus usually has three parts, the address bus, the control bus, and the data bus. Since RAM holds data, and CPU performs some action (processing) on the data, the CPU first sends and address to RAM by turning on (I use “lighting”) wires to indicate (in binary) an address number. It then uses the control bus whether it’s reading from, or writing to, RAM. Finally, if it’s reading from, RAM will send the data across the data bus. If it’s writing to RAM, the CPU sends processed data back to RAM using the same data bus.

A Cook in the Kitchen

I see quite a few explanations made on the relationship between CPU and RAM, usually they are likened to the human brain. I suppose that works, but what helped me understand the basics of this communication was thinking of a cook retrieving and following recipes in the kitchen. It’s an OK analogy, although you probably wouldn’t store food in a cook book. So pretend this cook’s cookbook is really magical.

  1. Get Recipe -> Address Bus.
  2. Read Recipe or Store Food -> Control Bus
  3. Follow Recipe or Place Food in Cookbook (magic) -> Data Bus

The cook will follow the basic 3 steps above. First she finds the Scrambled Eggs recipe by its name (they’re in alphabetical order, of course). Then she decides whether she wants to read the recipe, or store some eggs she just made according to the recipe. In this case, she’ll read it. Finally, she follows the recipe and makes Scrambled Eggs.

In my diagram of a simple CPU, the buses are arbitrarily 2 bits wide and the number of address slots in RAM are 4, mostly because I ran out of room and 4 seemed like a nice easy number. In reality, you would have many, many more slots and bus wires.

1. Address Bus

The CPU first sends an address to RAM using the address bus wires. Each wire represents a bit (a 1 or a 0). In this example there are 2 of them, for a total of 4 possible addresses (00, 01, 10, and 11). If the CPU wants the contents of RAM address 2, it would light up the first wire and keep the second dark to represent 10, or 2 in binary. RAM would send the letter “Picked”, as requested, across the data bus if the Enable wire is lit. Or if the set wire were lit, data would be written to RAM’s data in the 3 slot, perhaps something like “A Peck”.

2. Control Bus

This controls receiving and sending. Enable means “receiving”, set means “sending”. You can also think of this as read or write. In my simple CPU above, lighting the enable wire transfers data from RAM to CPU, lighting the set wire writes data already processed by the CPU to RAM.

3. Data Bus

This is where all the goods are stored in binary. I just picked some funny words to represent data because it seemed like a nice way to illustrate the point, but the data could be anything. Every kind of data out there has some interesting way of encoding it. For example, letters use a system called ASCII or Unicode to translate letters to binary. Pictures use a variety of formatting, like JPEG or PNG, etc. But it’s all binary in a computer.

What is a Computer?

My Elevator Pitch

Computers are devices built to perform computations based on data given to them, the grand majority of them using a base-two number system called binary. While human beings are capable of performing advanced calculation, we are error-prone and are unable to upgrade our organic hardware with newer designs (yet). Computers have been designed to be “programmed” or instructed to perform different calculations without having to be re-wired or changed physically. This design has allowed computers to assume responsibility for and transform a vast number of tasks, workloads, and services in human civilizations.

A Quick History of Computers

Update 04/15/2019 – My 5-year-old found The Story of Coding (affiliate link – supports my knowledge quest) at his school book fair. I was surprised to find it was not just a story of coding, but a history of computers, the Internet, the web, and code. It’s full of some of the same factoids I have laid out below, but with better pictures, structure, and concise prose. It’s written for children but I immediately took it to my wife (and later parents) and told her to read it so she’d know why I’ve been so excited and passionate about technology all these years. This book is great for kids but also great for someone that’s maybe slightly interested in technology and wants a 5-minute overview. Now back to my original post:

(Got most of this from shamelessly from Wikipedia, see sources below)

Early history is peppered with interesting archaeological artifacts that suggest humans use tools for mathematical calculations, starting with the Ishango Bone (20,000 B.C.E.) , to the abacus (2,500 B.C.E.), to the Antikythera Mechanism (125 B.C.E.). In addition to hardware, humanity achieves logical milestones before the common era such as the first use of binary in India, negative numbers in China, and the first computation resembling a “program” in the Roman Empire.

80-column punch card. Photo by Gwern derivative work: agr [Public domain], via Wikimedia Commons

Achievements in mathematics and engineering continue until some major breakthroughs in the 1800s – in 1822 English mathematician Charles Babbage proposes the “Analytical Engine”, along with his counterpart Ada Lovelace (English mathematician) who developed a method for the machine to move beyond simple calculation. She proposes a way for the machine to calculate Bernoulli numbers in what is arguably the first “algorithm” developed for a computer. In 1847 English mathematician George Boole develops Boolean algebra, a logic system that forms the basis of binary computers today. In 1890 U.S. Census employee Herman Hollerith founds the Tabulating Machine Company (now known as IBM) and develops a method to store data readable by machines, using punch cards.

ENIAC. U.S. Army Photo [Public domain], via Wikimedia Commons

American engineers John W. Mauchly and J. Presper Eckert construct ENIAC, a completely electronic programmable computer using vacuum tubes for computation. Over the next 10 years, many computers are built in various developed countries, including designs using transistors, a marked improvement over vacuum. A new concept is conceived of by American genius mathematician Grace Hopper – to convert English words into “machine code” executable by computers using a program called a “compiler”. Programming “languages” are created, including FORTRAN and COBOL. In 1958, the integrated circuit is invented at Texas instruments by engineer Jack Kilby and physicist Robert Noyce (who went on to co-found Intel).

The IBM 5100. Photo by Sandstein [CC BY-SA 3.0], via Wikimedia Commons

As computers moved to integrated circuits and thus smaller sizes during the 1960’s and 1970’s, many achievements were made: BASIC language was developed, packet switching and ARPANET are developed by ARPA, the Unix operating system and C programming language are created, Intel develops dynamic RAM chips, Ethernet (Xerox) and the TCP/IP protocol suite (ARPA) are created. In 1975, Bill Gates and Paul Allen create a microcomputer implementation of BASIC, and go on to found Microsoft. In 1976 Apple is founded by Steve Jobs and Steve Wosniak. In the same year, Whitfield Diffie and Martin Hellman publish the findings of their research into asymmetric cryptography in a paper entitled “New Directions in Cryptography”. In 1977 Ron Rivest, Adi Shamir, and Leonard Adleman create the RSA cryptosystem. Intel creates the first x86 microprocessor in 1978, the current global CPU architecture standard as of writing, 2019.

IBM PC 5150.
Photo by Ruben de Rijcke [CC BY-SA 3.0], via Wikimedia Commons

In 1981, IBM announces the IBM personal computer, and partners with Microsoft to create the operating system, MS-DOS 1.0. In the same year RFCs 791 and 793 establish Internet Protocol and Transmission Control Protocol, respectively, as open standards. Richard Stallman creates the GNU Project in 1983 as a free and open alternative to proprietary UNIX. The same year sees RFC 881 create the Domain Name System. Microsoft Windows is launched to little fanfare in 1985. To end the decade with a bang, in 1989 Tim Berners-Lee and his team at European Organization for Nuclear Research (known as CERN) develop a concept they call the “World-Wide-Web”, complete with HTML, HTTP, a test web server and browser client.

Apple Power Mac G4. Photo by baku13 [CC BY-SA 3.0], via Wikimedia Commons

In 1991 Linus Torvalds announces his new kernel on Usenet, the next year it is integrated into Richard Stallman’s GNU project while it waited for its own kernel to be developed. Over the next two decades, the operating system known as “Linux” would come to dominate many computing markets and pave the way for the proliferation for free and open source software. Linus changes Linux’s initial license to GNU General Public License (GPL). In 1993 the Moving Picture Experts Group (MPEG) designs an audio format for its MPEG-1 and MPEG-2 standards called MP3. 1995 is a big year – Sun Microsystems creates the Java programming language, Microsoft releases Windows 95, and Netscape announces Javascript development. In 1997 Deep Blue beats World Chess Champion Garry Kasparov in a chess match.

The HTC Dream Smartphone, running Android OS. Photo byAkela NDE [CC BY-SA 3.0], via Wikimedia Commons
The Raspberry Pi 1. Photo by SparkFun Electronics from Boulder, USA [CC BY 2.0], via Wikimedia Commons

In the year 2000, both Intel and AMD release CPU’s clocking in at 1Ghz. RSA security releases the RSA algorithm to public domain, making unbreakable asymmetric encryption and authentication algorithms fully available to the general public. In 2001, Mac OSX by Apple and Windows XP by Microsoft are released. 2007 sees Apple’s first iPhone. In 2008 the first version of Google’s acquired Android Operating System is released. In 2009 Windows 7 is released, along with the source code for Bitcoin. The Raspberry Pi, a device about the size of a credit card, is created in 2012 as an educational tool for kids to learn computer programming, for a price of $35.

What Computers Do

This is actually kind of an interesting question, with seemingly limitless answers due to a computer’s programmable design. My day is currently made up of many activities that are in some way assisted by computers. In my personal life I communicate with my family and friends using SMS, I keep track of events using my phone’s calendar, I search for tech articles on Google. My 5-year-old son likes Minecraft and Youtube. At work I use Microsoft Office 365 for email, and Cisco Webex for meetings. What I’ve discovered that those are just the uses, not what computers actually do. The answer is just one thing – computers follow instructions. In computer land, instructions are called code. This is not a coding blog, I just include some code here and there to illustrate a point. The “Hello World” program that kicks off any coding course really blows my mind when I think about it:

#include <stdio.h>

int main() {
    printf("Hello World!");
    return 0;
}

I can type a couple of quick commands (“helloworld” being a text file containing the above code):

make helloworld
./helloworld

I get this output on the screen (the same commands work on a Mac, or press F5 in Microsoft Visual Studio on Windows):

Hello World!

What is mind-blowing for me about the hello world program is the printf(“Hello World!”); line. I can literally change what’s in-between the quotes to whatever I want, and the computer will spit it out for me. I could put another phrase in there, like “Hello, I’m Johnny Cash” or “Computers are cool”, or something elaborate like the complete works of Shakespeare. The computer will work tirelessly and thanklessly for me until it completes its task. That’s some power.

The Current State of the Computer

You may be wondering, what happened in the 2010’s? Some of the coolest advances in technology have happened after 2010, surely there would be something of note on computing to put in the history section. From my point of view here on February 26th, 2019, advances in hardware and computational power seemed to have slowed compared to the decades before it. Perhaps this slowed growth is due to nearly every American owning a computer or some sort (desktop, tablet, phone, etc), according to Pew Research in 2018.

I’d wager that the major innovation is going not into individual computers anymore, but into to services (think Office 365, Netflix or Facebook). Gartner, Inc.’s 2018 survey for cloud computing points out that cloud services continue to grow by big percentage points, while desktop and laptop PC sales continue to slump, according to another Gartner, Inc. 2018 survey for PC sales. The growth of these cloud services are fueled by data provided by the cloud’s cousin, the Internet of Things. Gartner’s 2016 IoT survey tell us that in 2016 the number of things connected was at 6 billion, expect to be at 8.4 billion by 2018. I’m excited to see what future innovations will bring in the years to come.

Sources

https://homepage.cs.uri.edu/faculty/wolfe/book/Readings/Reading03.htm

https://en.wikipedia.org/wiki/Timeline_of_computing_hardware_before_1950

https://en.wikipedia.org/wiki/Timeline_of_computing_1950%E2%80%931979

https://en.wikipedia.org/wiki/Timeline_of_computing_1980%E2%80%931989

https://en.wikipedia.org/wiki/Timeline_of_computing_1990%E2%80%931999

https://en.wikipedia.org/wiki/Timeline_of_computing_2000%E2%80%932009

https://en.wikipedia.org/wiki/Timeline_of_computing_2010%E2%80%932019

http://www.pewresearch.org/fact-tank/2018/09/28/internet-social-media-use-and-device-ownership-in-u-s-have-plateaued-after-years-of-growth/

https://www.gartner.com/en/newsroom/press-releases/2018-09-12-gartner-forecasts-worldwide-public-cloud-revenue-to-grow-17-percent-in-2019

https://www.gartner.com/en/newsroom/press-releases/2019-01-10-gartner-says-worldwide-pc-shipments-declined-4-3-perc

https://www.gartner.com/en/newsroom/press-releases/2017-02-07-gartner-says-8-billion-connected-things-will-be-in-use-in-2017-up-31-percent-from-2016