The Real Zack Morris

An Open Letter to Apple on App Financing

Tue, 02 Oct 2012 15:39:53 -0400

Dear Apple,

With Amazon’s recent decision to begin lending capital to sellers, I thought it was time to suggest the possibility that you could do the same for your App Store developers.

It’s tremendously difficult for independent developers to raise funds for their projects. Many of us have gone for extended periods with little or no income, putting our savings towards finishing projects that often take months or years to complete. It’s notoriously difficult to raise funds for shareware games and other small projects. Banks scoff at our business proposals. Most of us don’t know any angel investors outside of immediate family. There is always Kickstarter, but successfully funding a campaign can be elusive. That leaves credit cards, a financial trap for developers unlucky enough to find themselves in the long tail of a product’s sales, where money trickles in too slowly to keep up with interest.

Meanwhile Apple has over $117 billion in cash reserves. With an average project cost of say $25,000, Apple could fund over 4 million developers right now. If you want to think different and grow the app market, that would surely do it. You could use your iTunes Connect financial reports as a kind of credit check to determine if a developer’s track record makes him or her eligible for funding. You could even have refinancing options if a second investment round is needed, where developers could trade some of their payout percentage for additional funds, or sell ownership of their app to Apple to close out a loan that’s looking at a long term repayment.

With the economy moving more towards services, we need to think about shifting investment from inventories to human capital. I was close to finishing a major game project for the iOS App Store but ran out of money and recently decided to take a contract at a company instead. If sales would have been anything close to our run in the Mac App Store, Apple’s 30% cut could have potentially been in the tens of thousands of dollars. Multiply this across the many thousands of struggling developers, and you are losing out on millions of dollars. It’s too late for me, but others could benefit from your investment, and you could increase your profits in the process. I hope you’ll consider it.

Sincerely,

Zack Morris

The Slowness of Virtuality

Thu, 13 Sep 2012 21:17:16 -0400

I need to get something written, so I was just thinking how agonizing it is to draw something like a conic section in Google Sketchup. I finally got it to work, but it was so much slower than just using a lathe in real life.

It got me to thinking about how slow it is to do anything on the computer. In the old days, if I wanted to test out a little transmission with gears in Legos, I just built it. It took seconds sometimes. But to do anything on the computer takes somewhere between a day and infinity, depending on if I can even get it to work or if the scope grows beyond the original parameters.

It’s gotten so slow for me to even work anymore, that paradoxically I haven’t gotten anything done in years. We remodeled our house. We landscaped our yard. I built patio furniture for my sister. But I still can’t get OpenGL to perform properly on iOS, or write even the simplest blogging script with Ruby in Jekyll.

In a way, I think technology has reached a crossroads where it’s beginning to undermine the economy and our productivity. Sure, we can enter thousands of numbers in Excel and a number pops out. But we can’t do rapid prototyping or visualization. We only think we can. Pioneers raised barns in a day. We can’t even raise funding.

I feel like technology needs a reboot. A total rejection of the appearance of progress. I’m not exactly sure what the future is going to look like, but it’s going to be a lot more like Legos than Google’s Go. It’s so self evident to me, so blatantly obvious, and causes such a violent reaction from technologists, that it almost has to be true. It’s one of those ideas that looks stupid at first glance but has the potential to change the world. Exactly the type of idea that hackers and startups are looking for.

I can safely assume that nobody is going to do it. I would love it if someone did. But since they won’t, maybe I just will. If only I had $500 for rent this month. Oh well, back to the grind I guess.

A Mistake in Go

Sat, 07 Jul 2012 11:57:00 -0400

It’s been 3 months since I last wrote, now that freelancing takes 100% of my time. I’ve fallen into the busy trap.

So I decided to post something, anything, to get past my writer’s block. I was reading this jgc article about Go and it all sounds about right to me, and I think very highly of Go because it breaks the cold hard strictness of other languages to give us the leverage we need as programmers to get Real Work Done.

However, I think I found the first real chink in the armor of the Go philosophy, and I haven’t even used it yet. They think there’s a “One True Way” to format code: gofmt. I tried the example code, and sure enough, it’s not how I write code. That means that the One True Way contradicts 25 years of experience. Not good.

Try going to http://golang.org/ and push the Run button. Pretty cool, right?

Now put a newline before the “{” so your code looks like:

func main()
{
	fmt.Println("Hello, 世界")
}

And push Run. Oops! I get:

prog.go:8: syntax error: unexpected semicolon or newline before {

What the heck is going on here? I tried a few other tests with if-commands and sure enough, the { has to go on the same line as the statement.

This might not seem like much of a problem, but let me give you a couple of examples of operations that I do dozens of times a day sometimes:

///// force execution of a block /////

//if( x < y )
{
	fmt.Println("Hello, 世界")
}

///// try a function block in another function to find bugs /////

func test()
{
	fmt.Println("Test")
}

func main()
{
	//test()
	{
		fmt.Println("Test")
	}
	
	fmt.Println("Hello, 世界")
}

///// try variations of parameters or return values /////

//func bigmess(x int, y int, a char, b char, c char ) int
//func bigmess(x int, y int, a char, b char, c char ) char//int
//func bigmess(x int, y int, a char, b char, c char, d char ) int
func bigmess(x int, y int, a char, b char ) int//, c char ) int
{
	return x + y
}

If I had to go in and micromanage the newline and { every time I wanted to test execution of a block, it would be a huge waste of my time.

I think there are two reasons why Google decided to criminalize this type of behavior. The first probably has something to do with their semicolon emission code. Maybe the parser has a hard time figuring out the syntax for certain kinds of whitespace, so acts a bit like Python.

The second is what I think of as preemptive retaliation, or pretaliation. This was the philosophy used in the Iraq War to bomb the $#!@ out of “terrorists” so that they can’t bomb the $#!@ out of us. Except it’s overzealous to the point of being harmful. Google thinks that if we comment out things like if-commands, that we might forget and leave the comments in production code. They are protecting us from mistakes that beginners make. A variation of this is:

if( x == 100 )
//if( x = 100 ) // still compiles, just not as expected!

vs:

if( 100 == x )
//if( 100 = x ) // won't compile

Like defensive driving, the second case is safe in case someone slips and forgets an equals sign. The thing is, I haven’t made a mistake like that since the 90s. I also sometimes write code like this in c++, which is perfectly valid:

while( (myPtr = GetNextItem( myCollection )) != nil )

The cost of making every_single_if_command in my code read backwards is too high for me. I’d rather be free to explore the problem space in my own style, errors be damned. If I spend all my time defensive driving, then I have no time for my self conscious to solve problems for me in the background. This is uber-authoritarian to me.

A better way to handle this is to turn on warnings and clean up your code so that there are none. The compiler will warn you about “=” typos. But most of us are too lazy to fix a huge codebase when we upgrade compilers and suddenly get 1000 warnings that are generally overreactions. But the warnings are there for a reason, and your life will be easier in the long run if you deal with them now and understand why they are there.

So what’s the solution to forgotten if-commands? Version control. It’s a PITA to set up, but everyone should become familiar with git/mercurial/svn and use them even for their personal development. Proofread every difference when you check in your code or check out someone else’s. Nine times out of ten, these testing comments jump out at you. You should also become comfortable with the idea that all code has bugs, so that when they happen, you have a bug reporting route in place for users.

If I had been on the Go development team, I would have focussed more strongly on this meta level of code, and thought of clever ways for the computer to infer what you are getting at, or warn you that you are going down a wrong path before you hit a dead end.

Maybe I would have invented a silent comment, something like the “#” sign in shell scripts, that would be used specifically for commenting out code that you expect to keep. It would run the code in production mode, but not debug mode. When you hit the head scratching bugs, you could look for the silent comments and find what you forgot to put back in.

But I would never stop you from being you. Or penalize personal style because it’s in the minority.

I’m right about this. I’m nobody, so they’ll never fix it. But I’m right.

Solve My Biggest Problem and I'll Solve Yours

Wed, 28 Mar 2012 16:41:00 -0400

I have not written for some time. For the last year, I’ve been freelancing online and fixing old Macs to bootstrap my shareware business. It takes all of my time. Here is a table of the startup process, as I’ve experienced it as a “technical cofounder”:

Stage			Effort	Timescale
----------------------------------------
Discovery		0%	1 day (hackers know what to do)

Prototyping		5%	6 months to one year

Customer acquisition	5%	6 months to 2 years

Bootstrapping		90%	years to decades
 - Day job		10%
 - Freelancing		10%
 - Odd jobs		10%
 - Beg, steal, borrow	10%
 - Trade health		10%
 - Pull all-nighters	10%
 - Put dreams on hold	10%
 - Live sub-standardly	10%
 - Postpone social life	10%

Expansion		0%	first world problem

For me, the elephant in the room is bootstrapping. It just totally dwarfs every other problem by at least an order of magnitude. I think the reason that 90% of businesses fail in the first year is that they are either unwilling or unable to do what it takes to support themselves while the business gains traction.

This problem is compounded by the fact that our western economic system has no system in place for micro-investing. Nobody today - no rich friend, no group of investors, no bank - is going to give you the $50,000 it takes to live for two years while you build your business.

Even if you manage to get the money like I did (by walking into a bank at the height of the credit bubble in 2005), lenders will want it back when your business fails, like mine almost did in 2007 when my lines of credit tightened before the housing bubble collapsed. This will be extremely high interest debt, so you may find yourself working at a job that makes you miserable for a few years afterwards. It’s like failing twice.

But say you make it through all of this and your business is earning you a few hundred dollars per month and you can make ends meet with a gig or two every few weeks. You can see trends in your business, like if you write another app or do some cross promotion, you can double your income briefly in the short term or raise it gradually in 3-6 months. But somehow you have to keep raising money month after month to afford basic necessities like rent/food/gas to sustain your meager burn rate. This takes all of your time, which prevents you from putting in the work required to grow. Which subsequently causes friction between you and your partner, your significant other, etc. What do you do?

I would love to be able to encourage everyone to quit their job and go on eLance to gain the freedom to do what they really want to do, but unfortunately I can’t. So far I’ve only averaged about minimum wage from the mainstream sites. I have high hopes for the startups coming online though:

http://grouptalent.com/

http://hirelite.com/

http://coffeeandpower.com/

http://gun.io/contracts/

http://hackertrade.com/

This is still an open problem. It falls under the category of “nobody’s solved it because if they did, everyone would be using it!”

It won’t be solved until it’s solved FOR ME.

I’ll know it’s happened because I’ll have gone onto a website like kickstarter and gotten as much money as I needed for my business. I’ll have solved the simpler problems like gaining traction, because I’ll have had the social proof that what I’m doing is desired in the world. I’ll know it’s solved because unlike the last few years, I’ll be doing something besides bootstrapping.

It’s really a lot like dating. I remember when I could never meet girls because I was trying too hard and they saw right through it. I remember thinking that it was like being in the doldrums, surrounded by water but dying of thirst. I went through a period where I decided to hack my behavior. I read the pickup guide and tried dating sites and forced myself to talk to random girls in places like grocery stores. Eventually something clicked and I transitioned to not really worrying about it, and a few months later met my girlfriend.

Fixing dating sites is still an open problem, but I wonder if there’s been a cultural shift in how people interact. A person today has vastly more resources at his or her disposal than someone trying to date in the early 90s before the internet went mainstream. Singles have what they need to better themselves and shift the problem from acquiring a mate, to becoming more desirable and entering into equal partnerships, eventually forming stable relationships and families.

That got me thinking about what’s really going on here. Why is our culture so dead-set on stopping entrepreneurs? Why all the hurdles and closed doors? Why did we have to wait till 2012 to even entertain the idea of government-sanctioned micro-investing?

There are the old arguments about the man trying to keep us down, to provide cheap labor for a transnational corporate machine. Or perhaps not a large enough segment of the population wants to be entrepreneurs, so there is a natural bias against them.

But it goes deeper than that. I remember thinking how silly it was that the drinking age was 21 in college or that recreational drugs were illegal. After all, people were going to war and voting at 18. Surely the notion of a partial adult is illogical.

But think about it from a meta-level for a moment. Why make it so hard for young adults to recreate? What is the cultural goal here? It finally hit me that the struggle is by design. It’s important for young people to socialize with older people and get them to buy beer for them. Doors open in the process. The exciting world of adulthood has leaps of faith at its foundation. We’re training youth to form social networks where they can score drugs or get into parties and eventually meet future lovers or get jobs through friends. We’re training them to eventually be celebrities or executives or even politicians.

In the same way, bootstrapping is a rite of passage where entrepreneurs learn what customers want and how to go from begging for work to mutual exchange of value.

As a hacker living in my parents’ proverbial basement, I did everything I possibly could to avoid this process. I was basically standing against the wall in a high school dance, making excuses as to why the whole thing was ridiculous. I talked like I was the CEO of a major corporation and then went home and played nintendo. I was a loser.

Fixing bootstrapping is still an open problem, but there are shifts happening to make the process less painful. Consider an analogy. When I was in high school, I didn’t know any girls who had computers. Cell phones just weren’t around yet either. If I had seen the year 2012 at that point, with girls tweeting from their smartphones to their friends to come check out parties, I think it would have completely blown my mind. People growing up with the internet must have a hard time imagining the isolation that gen-xers felt growing up. They have new problems that I frankly don’t even have a frame of reference for. And that’s a really good thing.

Instead of finding the magic bullet for bootstrapping, I’ve decided to take a leap of faith. If you are a well-to-do entrepreneur or investor and would like to collaborate, message me on twitter or email me at zsm at inbox dot com. I’ve done a few freelance gigs for iOS but I’ve found the work to be entry level. I’d prefer to collaborate with clients to make game-changing apps and eventually improve the bootstrapping process for everyone. Raising funds is as hard for me as programming is for you. The opposite is also true. Perhaps we can help each other out.

Discuss on Hacker News

Functionless Programming

Tue, 07 Feb 2012 17:32:00 -0500

In my last post I gave my take on why every API is flawed. I realize that sounds defeatist but consider that it’s only the tip of the iceberg. There are a multitude of problems with programming today, and even if we set out to solve each one, we have to consider that the rest of the world isn’t likely to follow our example. So instead of changing the world, I want to talk about how I would like to change my own approach. If it’s good enough, who knows, maybe it will catch on someday.

Think back on your life. When was your period of highest productivity? Mine without hesitation was when I was using HyperCard at 12 years old (a language very similar to Applescript but not crappy). Here is a snippet of HyperTalk:

on mousedown
	domenu "New Button"
	set the name of button 1 to "OK"
	show button 1 at 100, 200
end mousedown

Notice that there is no real punctuation besides the occasional quotation mark or comma. This was the closest I ever came to natural language programming. It’s really pretty astonishing how quickly people grokked this language. My dad programmed a bit in it and I don’t think he had ever studied any other programming language before.

I’m not going to break down the code very deeply, because it’s obvious what it does if you have even the slightest bit of context. Most of the functions in HyperTalk are written as callbacks to messages. So they look like: on mousedown, on keydown, etc.

I’d say the only complicated piece of code here is domenu, because it’s calling HyperCard’s human interface to create objects. Think about that for a moment. You can tell HyperCard to do the same thing a person would do to perform the same task. That’s unheard of today. We’ve lost the ability to write macros in most languages, and I find that very sad.

Since it’s the only button on the page, we can refer to it as button 1. We could also say:

set myButton to it

Which means to create a variable referring to the result of the last command performed. Then we’d say:

set the name of myButton to "OK"

The show button line is fairly obvious. We’re in a cartesian coordinate system, so show the button at a 2D location like it’s on graph paper. In HyperCard the origin is at top left and increasing right and down, but it could be arbitrary if we were generalizing the language for OpenGL or the web.

***

I had a whole rant here about how I ~~hate~~ dislike Objective-C and other trendy languages/APIs that I decided to leave out since I’m talking about methodologies, not languages. But I don’t want to leave anybody out. I tried learning JavaBeans and was never able to. For some reason they are just too dang complicated. Same thing with Microsoft Access. It’s a cryptic environment where everything feels hidden and I constantly have to hop on the web to do even the simplest thing. Content managers on the web generally suck, XML files generally suck, really 90% of everything sucks.

So let’s get to the meat of this post. All APIs are flawed because you have to rely on some authoritarian decider to provide you some interface to the code they are proud of. This is a level above where we want to be: full control. The best way to provide full control was developed half a century ago - the unix console. Things like streams. Those are actually a level above what’s really going on, which is black boxes that take some input and provide some output. Functional programming is the syntax above this base level.

Let’s think forward some arbitrary period of time and imagine how programming works in Star Trek or Star Wars or your favorite sci fi universe. We can think of this as manipulating state by controlling aspects of a block of data. All we are trying to do is get the computer to do what we want. Today we have a world where Scotty talks into a mouse and nothing happens.

But we know that what really happens is programmers spend a couple of hours in google and Stack Overflow until they find the snippet they need to do whatever trivial task the API makes difficult. Let’s start with this post, which I saw the other day on Hacker News. In the future, programming will be collaborative, whether it’s between users or artificially intelligent computer agents. Right now we spend 90% of our time formatting our answer to fit what the computer can understand. In the future, we’ll spend 90% of our time reviewing the options available and then choose the one that best furthers our goals.

Every metaphor will be a static process. We’ll feed it some data and in an imperceptibly small period of time we’ll get the result. We can think of each box as a leaf in a big tree that under the hood is represented as a hash that distinguishes it from other objects, based on the inputs and outputs it’s defined for. Let’s talk to this futuristic computer in our text-based dialect:

make a screen

Result:

make:
	1) verb. instantiate something. Associated with imperative programming.

a:
	1) superfluous word.
	2) Sometimes used to name a variable.

screen:
	1) noun. usually a graphical user interface to view state.
	2) verb. filter results.

please choose one of the following examples or write something similar:
	1) screen1 = screen
	2) set myScreen to screen
	3) (screen (dimensions 0 0 1 1))
	...

So the user tries one of the suggestions:

Result:

screen1 = screen

A new screen pops up with default dimensions. from here, maybe the user tries to make a button:

make button in the center of screen

Result:

make:
	1) verb. instantiate something...

button:
	1) human interface element to trigger an action from a click.

in:
	1) verb...

the:
	1) superfluous word...

center:
	1) noun: indicates the spatial center of an object.

of:
	1) verb...

screen:
	1) possible reference to screen1
	2) noun. usually a graphical user interface to view state.
	3) verb. filter results.

please choose one of the following examples or write something similar:
	1) button1 = button( screen )
	2) set myButton to make button in screen1
	3) (button (dimensions 100 200) screen)
	...

So the user tries one of the suggestions:

Result:

set myButton to make button at center of screen1

A button appears in the center of the screen.

And so on and so forth. Now the remarkable thing about all of this (I know this sounds weird) is, none of these types are named in this futuristic OS. Everything is a concept, based on the hash of the inputs and outputs that describe what it does. The computer infers the types, names and actions that the user is trying to perform. The questions the user asks will be stored as part of the program to explain the syntax that evolves, which itself will have a large degree of uniqueness but will still share metaphors common to other programs.

It also infers the methods of programming that the user is comfortable using. Some users like algebra, some don’t know what it is so prefer natural language, some prefer c-like or lisp-like syntax. It’s fine to mix any of these, because everything can be represented as nouns and verbs under the hood, or matter and energy, or however else we like to visualize it.

There is no API to speak of, other than the console. The API is actually a kind of wiki/chatterbot on the order of say kilobytes to petabytes that represents the relationships represented by the current programs in existence at the time the snapshot was taken.

Also since all of this is black boxes with some input and output, everything is actually a noun. Like I said, this program is represented as a tree of hashes under the hood, with hash values drawn from the current wiki snapshot. The wiki itself even has a hash. I say hash but what I really mean is a relationship graph, which is lisp.

Only this is a special kind of lisp that can do things like trigger actions when other values change. Users will tend to tie actions to the state of other objects without considering verbs, because that’s how the real world works (like how you set an item’s location in javascript and it just moves without explicitly calling a function to set its location). I like to think of this as functionless programming. When users create relatively advanced spreadsheets in Excel, they are just typing the equals sign and clicking on boxes, with the occasional math operator between them. I don’t think that people see these as functions (even though you and I know they are). They see them as triggers. And how much more productive is that than whatever crappy environment we have to use right now? I think that we’ve been forced to work at such low levels for so long, that we haven’t exercised the parts of our brain that will allow us to really take advantage of computation the way non-programmers will when they discover they can have the computer do something for them when something else happens. We’re seeing this right now with some of the websites that let people chain tasks from other websites together.

Notice some interesting features of all of this. It loosely represents the point and click metaphor of the internet. The user can always ask the computer for more elaboration or more suggestions. Programming will become an interactive exercise where the user is creating code with multiple choice instead of paragraph form.

The user will also be able to save new code as a hash based on the original hash and the new relationships, and then share it with the world, to be integrated into other APIs. The hash will also act as a repository, and since the relationships are spelled out explicitly, APIs will be able to share code with each other (that replicates other inputs and outputs but with smaller code or faster speed) creating new APIs. This will very quickly simulate many aspects of sexual reproduction.

Humans will be able to act as APIs so multiple humans can share connections to an API that is evolving. It will be valid to ask another API (possibly a human) to suggest code that performs a function. Those new inputs and outputs can then be integrated into the API.

It will not take long at all for APIs to become sophisticated. Constant evolution will generate APIs that are so sharp that it feels like talking to a human. The suggestions will be relevant and nearly always correct. Notice the line where I said “1) possible reference to screen1”. The API itself will evolve to suit the user, as he or she uses it.

From here on it’s all just speculation, but I think that what will happen is programmer ability will not be as important as it once was. The “best” programmers will be able to just jump in and tell the computer what to do with very few questions, but with so many algorithms cached in the API, they won’t tend to come up with much novel code. The computer will notice what they are doing and suggest the prior art. Software patents won’t exist, unless you want to talk about patenting a hash representing the whole context of a program and its root API. Change one bit anywhere though, and you have a new hash, so new patent by definition. By this point we’ll have algorithms building our houses and growing our food, so without a money system, maybe patents won’t make any sense anyway. They don’t use money on Star Trek. But I digress.

***

So how does this help us today? Well, IMHO we have a lot of good metaphors but crappy implementations. I’m basically talking about BASH files with agents that help the user explore the problem space, but approachable by mere mortals. UNIX has a bunch of holdovers from when memory and disk space were scarce, like abbreviations and cryptic operators that make it difficult to convey ideas. That will largely go away when someone can read a program in natural language from the beginning.

Programs of the future will just be language-agnostic text files where users embed whatever languages best solve the problem, as lines similar to BASH files. They will probably end up looking like a transcript from the programming session. Or in this case, just the summary:

ABCD1234ABCD1234

make a screen

...

1) screen1 = screen
2) set myScreen to screen
3) (screen (dimensions 0 0 1 1))

1

make button in the center of screen

...

1) button1 = button( screen )
2) set myButton to make button in screen1
3) (button (dimensions 100 200) screen)

2

This code would show a new screen with a button in the middle. The first line is the checksum encoding the API and all of the context needed for this code to be parsed and executed. It holds the summary of the conversation that created the code we are looking at. This is just one level of detail. The user could zoom in further and see full descriptions of all the words, or zoom out and see just the pure code. Each line is whatever the user would have executed on the console. The point is, what we think of as the end result code, language or syntax isn’t the important part. The logic and context are. And they can be presented any number of ways, generated from the user’s chain of thought.

I’m glossing over a ton of details obviously like how the runtime will work. But remember that that part will be provided by the API. It has enough context to understand what the user is trying to do from just this small bit of code. If users want to get explicit and create a process with some amount of guaranteed memory and access to various external devices, they could certainly do that. But there won’t be much reason to, because that infrastructure is explicitly provided by the API.

So it won’t make any sense to write API descriptions in archaic formats like Objective-C or .NET. It will be better to use flat, simple descriptions in human readable formats more like HyperTalk or JSON or even SQL. We’re going for syntax-less, stateless descriptions like static websites so users can always understand what they are seeing and dive down further when they are ready.

Here’s one concrete example of what I’m talking about. Right now to talk to a scanner, you have to use some archaic driver and a bunch of weird commands to control it at too low of a level to be useful to most people. In the future, you’ll just type something like “scanner” on the console, the API will infer that you are asking about the scanner and tell you all the inputs and outputs, and then you’ll type something like “scan A4 to myfile.jpg at 300 dpi grayscale” and the API will figure out what you want and make the file. You can always try again in color or with a different colorspace or whatever else you want, as you dive further into the available options. No drivers needed, because your code interfaces the API in the scanner directly, just like if it was a website you were pointing and clicking on. Cool!

But remember that there are truths to programming that will never go away. People will always do things their own way. We’ll probably never get past the strict/unforgiving APIs that operating systems and embedded devices or peripherals use today. But with a pure middle layer like this, we’ll be able to treat the relationships as the real OS and their crappy APIs as any other black box. If a process like BIND or Apache crashes all the time or has to be sandboxed, then that description will be encoded in its logic hash. Think of it this way. If the user has to check for an exception or failure code from a process, then that bit of crapulence will get encoded just like any other logic.

But many of the smaller algorithms will be provably error-free. As they evolve, then the seemingly intractable tasks of making things like web servers will begin to become commonplace. By that I mean, kids that can read will be able to write web servers. And they will.

It won’t seem unusual to write a program that does 90% of what commercial programs like Photoshop do in a few minutes. Just create a screen, create brushes, create layers, create special FX to act on each pixel, and before you know it you have a functionally-written app with no real main loop, that can’t crash, and which can be trivially extended.

This seems impossible until you consider leverage. You don’t just go and write something like Photoshop as a pure console app. You write it in something like Qt or above the web browser that comes as part of your API snapshot. Only in the future, the APIs will be much higher level than even web browsers today. They’ll work more like Apple’s Siri. They will always be bending over backwards to do exactly what you want. Someday they’ll say computers are man’s best friend, not dogs.

This all goes beyond what I could ever do right now, but I don’t think a bare bones implementation of this is actually that complicated. It will honestly look like a wiki with a text prompt. I’m not really sure what I would do with it, because I can already program. But I’m very curious to see what children come up with. They will want to write their own versions of anything that costs money. Maybe that’s a hint right there.

Discuss on Hacker News

Every API is Flawed

Fri, 03 Feb 2012 14:00:00 -0500

Apologists annoy me. As hackers, we make all kinds of excuses for why things are a certain way, and then we route around the damage and encapsulate it away inside our own APIs. John Carmack once said (as I recall) that if a system API is good enough, then we don’t need a library. Libraries (especially open source) exist because of fundamental flaws in operating systems that make them difficult to use. Look around. Libraries are everywhere. Systems suck.

There’s a few reasons for this, but the main one is that since everyone has a different life experience, there will never be a consensus on what should be standard. Some people want speed, some people want readability, some people want small code size, and others like me want it all. I’m always searching for the latest and greatest methodologies.

The second reason, as I see it, is proprietary software. Companies have a financial incentive to create their own ecosystems. This runs counter to trying to create standards. Just look to Objective-C on iOS or how Android originally used java or all of Microsoft Windows to see this in action.

The third reason is human fallibility. No matter how hard you try, your software will always suck upon closer inspection at some future date. This is because technology is always improving, and someday software will be reduced to the minimum number of bits needed to encapsulate the idea that you tell the computer with your voice or brainwaves. The million lines of code limit in large projects is not so much a limitation of the human mind, but the limit of methodologies. Code grows because operating systems evolve and force us to include legacy code. It grows because new eyes look at it and we can’t include our memories with the comments. It grows because methodologies change.

But what really annoys me about all of this is when easy mistakes get lumped in with hard ones. I respect that no API is perfect, but what I have no patience for is programmers who lack the ability to simulate how people will use their code in the real world. When I’m finding major bugs 5 minutes into using an API, then the whole thing is suspect. It’s like trying to read a book written by a child. It’s quaint when the kid is just learning, but when it’s a child emperor and your life depends on telling him what he wants to hear, diplomacy becomes a full time job.

Welcome to mobile app development today. The internet bubble of the 90s had a certain quaintness because it was all so new. But now we are hitting the walls of entrenched dogma. Smalltalk was a cute language that brought some interesting ideas to the table. It’s kind of a half-lisp with its most important features being named parameters and message passing. Unfortunately it never caught on because named parameters turn out to be something that the IDE can provide for you by inference (so there’s really no need to make the syntax more verbose), and message passing turns out to be very difficult to debug, especially with concurrency (something you find out late in the game, which has sparked a debate between the node.js and thread camps). Since Objective-C is a hybrid of smalltalk and c, it inherits the worst features of both languages without bringing much innovation to the table. I think of it more as syntactic sugar than a new language. It was already showing its age by the mid 90s. But it’s somehow the language of choice for iOS development.

You’re probably wondering what any of this has to do with the two images at the top of the page. Well, the code snippet below loads a CGImageRef from disk in iOS. It’s one of the smaller c++ snippets I could come up with, without using an external library. It returns a ptr to the pixel buffer, that can be disposed with free(), and optionally tells the width and height if they aren’t nil. The very first thing I tried with a transparent PNG didn’t work (that’s the first image above). There is a bug in CGDataProviderCopyData() that cause any zero alpha pixels to come out white. That is the code that is commented out. The fix is to manually create a CGBitmapContext and draw the image into it translucently.

Now, I can already hear the apologists telling me that I shouldn’t refer to the raw pixels anyway, because of issues with colorspaces or whatever. Maybe there are problems with resolutions or any number of reasons to draw into a context. Honestly I don’t care. When I ask for the raw data, give me the raw data. Apple actually had to do extra work to do this wrong. My guess is that they allocated a white buffer under the hood and then decompressed the PNG over, skipping fully translucent pixels (as an optimization).

The reason I’m writing this post is because I encounter problems like this NEARLY EVERY TIME I TRY TO DO ANYTHING. In all APIs. Everywhere. I’m not even going to bother reporting this as a bug to Apple because 1) they won’t fix something this entrenched and 2) what I’m complaining about is a problem with THEIR methodology, where they discount developers’ time. They take the time to study how users interact with technology, but not developers. They are punting on this stuff far too often. It’s gotten bad enough that 90% of my workload now consists of working around flawed methodologies instead of writing useful code. I’m coming to realize that this transcends language or operating system. This is a problem with authority mainly, some stubbornness and even ignorance.

Nobody seems to have come up with a good way for APIs to communicate with each other, and I feel that it won’t matter how far languages progress until we deal with this issue. I have an idea how to solve this in my next post. I think the key is to not use language at all, but reexamine the metaphor of an API itself. Since APIs are guaranteed to be flawed for the reasons I talked about, let’s accept that and work on encapsulation. Let’s replace authority with merit.

#include "CoreGraphics/CGDataProvider.h"
#include "CoreGraphics/CGImage.h"
#include "CoreGraphics/CGBitmapContext.h"

void*    LoadImage( const char *path, int *width, int *height )
{
    CGDataProviderRef    dataProvider = CGDataProviderCreateWithFilename( path );
    UInt8                *pixels = nil;
    
    if( dataProvider )
    {
        CGImageRef            image = CGImageCreateWithPNGDataProvider( dataProvider, NULL, false, kCGRenderingIntentDefault );
        
        if( !image ) image = CGImageCreateWithJPEGDataProvider( dataProvider, NULL, false, kCGRenderingIntentDefault );
        
        if( image )
        {
            /*// this doesn't work because of a bug in CGImageRef that renders a PNG image incorrectly so that zero alpha pixels turn out white
            CFDataRef            data = CGDataProviderCopyData( CGImageGetDataProvider( image ) );
            
            if( data )
            {
                int        length = CFDataGetLength( data );
                
                pixels = (UInt8*) malloc( length );
                
                if( pixels )
                {
                    CFDataGetBytes( data, CFRangeMake( 0, length ), pixels );
                    
                    *width = CGImageGetWidth( image );
                    *height = CGImageGetHeight( image );
                }
                
                CFRelease( data );
            }*/
            
            // draw the image transparently onto a CGBitmapContext
            *width = CGImageGetWidth( image );
            *height = CGImageGetHeight( image );
            
            pixels = (UInt8*) malloc( (*width)*(*height)*4 );
            
            if( pixels )
            {
                CGContextRef    context = CGBitmapContextCreate( pixels, *width, *height, 8, *width*4, CGImageGetColorSpace( image ), kCGImageAlphaPremultipliedLast );
                
                if( context )
                {
                    CGContextClearRect( context, CGRectMake( 0, 0, *width, *height ) );
                    
                    CGContextDrawImage( context, CGRectMake( 0, 0, *width, *height ), image );    // now pixels contains image pixels
                    
                    CGContextRelease( context );
                }
                else
                {
                    free( pixels );
                    pixels = nil;
                }
            }
            
            CFRelease( image );
        }
        
        CFRelease( dataProvider );
    }
    
    return( pixels );
}

Discuss on Hacker News

All In

Mon, 30 Jan 2012 17:30:55 -0500

I wake up as my phone rings and check the number. Another long distance area code, probably a debt collector. I push the button and set my head back down on my arms but realize it’s been twenty minutes so I should probably get back to work. The screen comes into focus, the first tab open to a protocol description for an encrypted p2p wireless network. Tabs cascade off the right side of the screen: exascale computing, crowd funding, 1% direct debit, quaternion interpolation, cheap graphene, I lose interest at seven distractions. Cat’s hungry, out of cat food. Need to take another test on eLance. Rear bike tire starting to fray. Roof leaking at sagging attachment. Did I shower yesterday?

New chat messages. Friend finished his contract, good for him. I still need to open my other friend’s TV to see if it has failed capacitors like the iMacs I’ve been fixing. Partner’s not sure if local contract is still available.

Email from girlfriend, three jobs available in my field. So tired. Shake it off, remember the goal.

Back to programming. Flip open Xcode. No gl_FragDepth() in OpenGL ES, can’t render pixels and shadows in one pass over the sky. Have to invent another way to do it. Can’t set stencil buffer value either. Who decided this stuff? Will have to encapsulate the flaws away, work around them. Do it right, make it readable. Later.

Hungry. Out of peanut butter. Out of bread anyway. Almost out of milk. Leftovers from yesterday, thank god. Yogi’s belly should be filled half with food, one quarter with water and one quarter with air. Wonder if philosophy was discovered through coercion or necessity. Irrelevant, it’s worked for a year now. Remember to learn yoga someday.

When was the last time I meditated? Three months ago? Strangely don’t feel the need to. Life finds a way. Less is more.

Why. The answers are elusive, ephemeral. When was the last time I felt certain? Maybe the summer of ‘93? Before divorce, college, alcohol and the working world? Or was it the fall of ‘99, in that brief interlude between college graduation and the end of the world? Or the summer of ‘01, just before 9/11? Was I certain at any time after that? What does certain even mean? Can a certain person be considered sane?

Faith was an alien word that never held any meaning until this year. I know all about reality. A person is capable of almost anything when determined enough. The universe can be bent around a will. There is before and after. Before one discovers the exacting nature of the universe, life is filled with possibility and the wonder of the unknown. After one has made a determination of the nature of the universe, of right and wrong and the proper way to live, and the universe provides that reality, does wistfulness of the way it was before set in. Power is emptiness. Control overrated.

But faith - the why, the unwritten word, the zen, that’s the final frontier. Seeking meaning, not having it. Walkabouts and vision quests and finding oneself. How does one compress a thousands desires into one day? Impossible. Solving every problem, inventing every device, demonstrating worth, would require immortality. What’s left is the hollow realization that it won’t come true. That the ideal world that would free us from the strange misery we inflict on each other will not only not exist anytime soon, but that even if it did, we would be the less satisfied for it. There are only small moves. Only dignity in living with our imperfections. The knowledge that we all fall, no matter how high we climb. The liberation of knowing that it does’t matter.

On to step two. I’m awake now. Averaging the whole of the world’s expectations blurs into random noise. Why am I here? How can I best ask that question, knowing that there is no answer that would make any sense to anyone else. Where to begin? What am I to do? What should I be doing RIGHT NOW? Not procrastinating, that’s for sure. Or is it? Not sure, we’ve established that. This is the moment, it’s up to me what to do with it. Procrastination is just as relevant as not eating or not driving or not talking. Take the time to imagine, see it through. Remember the goal: to let go, to make the world real again. You are not in control. Believe it this time. Let it be.

Just be. Take a deep breath. Stop thinking. Listen. The purring cat. The sleeping dog. The furnace kicking on. The computer fan. The fridge. The fingers on keyboard, the clammy feet, the kink in the neck, the spoon in the parts box, the eyeglasses under the junk mail, the forgotten projects scattered about the room in disarray. The spine tingling endorphins for no reason. The quiet inner voice. The day and the night and the sun and the moon and everything bigger than us. The jet plane in the sky. The ground beneath our feet.

On to step three. There’s no step three! I remember when that used to be novel. Now it’s adrift in a sea of the ever frothing foam of distraction. We are here now, let the memories pass. There is only the age old question. The unknown. There are the smaller questions, like, do we create this reality as we go? Did we choose it before we were born? Do we really have any say in things or are we just crazy enough to believe that? Does it matter? More distraction. So much distraction, everywhere, constantly pulling us from experience.

I grandiose my life, believing that I can tap into almost unimaginable resourcefulness. I could build sentient machines, I just need a hundred dollar development kit and a few years with free sample chips. I could build a solar device that costs next to nothing and could power the world for free, I just need a year of living expenses and access to the raw materials. I could build a flying machine that only needs the power to weight ratio of a human. And those are the easy projects. The hard ones, that keep me up at night, I’m not ready to talk about. The words of an insane person. Similar to the words of DaVinci, Tesla, Thomas Townsend Brown or Philo Farnsworth. Men far more brilliant than I’ll ever be, in times far more oppressive. Shame on me.

I’ll be embarrassed when I post this, because it’s too revealing. Too crazy, too fringe, too unrealistic. My dreams are my own because they push the limits of my abilities. My imaginary ones. The truth is, I can’t even hold down a job without feeling like a rat in a cage. I can’t even clean my office. I can barely feed and clothe myself. I know in my heart that most of my inventions won’t actually work. That they’ve been tried and there is some bit of physics that I overlooked, some complication that will make them too expensive to be practical. But I’ve spent my life knee deep in algorithms so complex that simulating them dwarfs the complexity of almost anything I’ve encountered in the material world. Surely a collaboration, with the materials and computation available today, could bring us something more useful than information technology.

I envy the John Carmacks of the world, the Paul Allens, the Richard Bransons. The ones who proved themselves financially and are now empowered to work on the world-changing ideas. I wonder what dreams they have. Not the mundane ones they fail at now like the rest of us do. The ones they only share with their closest friends. The ones that need a trillion dollars or the collective weight of the world to achieve. Immortality. Extraterrestrial contact. Free energy. Who knows. Who cares. We’ll find a way to achieve them, take them for granted, and move onto the next wonder until we have everything we need to endure until the end of the universe.

But then I remember, they are dealing with their own realities. They are every bit as trapped as we are, perhaps more so. “None but ourselves can free our minds” as Bob Marley put it. Their biggest worry is their own demise. Money can’t save them. And nobody is working on the real problems, so we’ll all go down. And that’s ok. Can’t opt out of reincarnation.

I write this for a selfish reason. To procrastinate fully, to rid myself of the need to daydream. Work has never been and never will be the problem. I solve problems handily, trivially. It’s the non-problems that get me. The problems of motivation, desire, engagement. It’s all such a waste of time. How do I trick myself into believing it isn’t, just long enough to make some progress? That is the real challenge I face.

Maybe I need a new technique to numb my mind as I live out a life devoid of achievement. Maybe the puritans who denounce alcohol, drug abuse and promiscuity in favor of blind faith are onto something. Maybe changing one’s mind is easier than changing the world. Maybe it’s possible to have a life of small dreams and then achieve them and feel good. I wonder what that must be like. I will probably choose that for myself in my next life. Maybe this is my first understanding of conservatism.

Or I need a machine that converts my efforts to money, like a treadmill. I could toil on it a fixed amount of time and get the money I require to exchange for the food and shelter that lets me live. We have this machine now, it’s called a job and takes half of our waking hours every day. I want to design one that’s more efficient. Real technology that allows us to get below the 2 hours of day labor that hunter-gatherers expended in survival. As the machine improves, labor drops to 1 hour, then a half hour, then 15 minutes and finally falls to a time period that can be combined with brushing one’s teeth. Then a machine can walk the treadmill and we’ll be free.

The why becomes the how. How do I make this happen, so that I can get back to asking the why? What is the shortest path? There is no large payout with a job in a short timeframe, so that’s out. There is the internet lottery, elusive but potentially lucrative. There is no in-between but people are trying. Should I put my efforts into helping to build this labor-saving device? Are there other people out there who have run through this same simulation in their minds, realized that the current reality isn’t getting them closer to their dreams, and yearned for something more?

I want out. I want to get back to the why. I have only small moves at my disposal. I have my spirit to sustain me. I have the now. The how is manifesting all the time. Someday I will have the who. I’m willing to do whatever it takes. I’m all in.

Homesteading

Thu, 26 Jan 2012 13:01:00 -0500

I read this concise response from Joel Spolsky the other day on the best way to divide ownership of a startup:

http://answers.onstartups.com/questions/6949/forming-a-new-software-startup-how-do-i-allocate-ownership-fairly/23326#23326

Discussion:

http://news.ycombinator.com/item?id=3489719

General points:

Startups are organized into pyramids with early arrivers receiving larger shares of ownership (Joel calls them layers, which work like stripes in the military - more seniority, more stripes).
Ownership is mainly decided by risk (the greater the risk, the greater the ownership).
The next deciding factor is monetary contribution (the greater the investment, the greater the ownership).
After that comes time (the earlier a person arrives, the bigger the slice of the pie).
Finally comes representation (the fewer people in your layer, the larger your share).

His main takeaway:

“The founders should end up with about 50% of the company, total. Each of the next five layers should end up with about 10% of the company, split equally among everyone in the layer.”

So let me get this straight. The two guys who started the company own half of it and the people who do the actual work get some fraction of however much is left, split over hundreds or thousands of people? Which gets progressively smaller as the company matures until after some point an employee can be reasonably certain he or she will receive stock options worth essentially nothing? And let’s not forget the investors, who own the other half we don’t see, whose workload consists of mainly skimming profits.

Yup, sounds like capitalism.

***

While I generally appreciate his insights and contributions to the tech world (I use Stack Overflow everyday), I find this post unsettling. It reeks of 20th century philosophy. Actually more like 19th. He did a brilliant job of summing up contemporary best practices to fairly compensate those involved. As long as those people are the founders, wealthy investors, or suck ups.

My main takeaway?

Don’t ever, ever, ever work for an established company. At least not one that used to be a startup. Which is basically all of them.

And not because of the inherent unfairness of the corporate structure. But because the best and brightest perpetuate it. Whether you work your way up through a company, find yourself in a position to start one, or even amass enough wealth to fund one, by the end of it you will almost certainly be feeding into the same mindset that virtually assures a difficult struggle and smaller piece of the pie for others. This is almost by definition anti-progressive.

My experience, the few times I’ve worked for others, is that businesses are kingdoms and employees are serfs. It doesn’t matter how skilled a person is, how creative, how motivated, effective or committed. In the end, your stake in the company works like binary. If there are 4 levels, the most you could hope for is 1/16th, split with a hundred other people. But really it’s worse than that. As far as I can tell there are really two levels, 1 and 0. The owners own 100% and the rest of you get, well, the rest.

Let’s take a typical small business doing $2 million in sales using a roughly 25-30% markup to bring in potentially $500-600,000. After taking into account returns, theft, breakage and other losses, let’s assume a 15% margin so $300,000 in cash flow to play with each year. Let’s say each owner invested $25,000 for 20% down on a $250,000, 10 year bank loan to start the business. There are all kinds of ways to arrange it but this is what I’ve seen for your average mom and pop shop:

Profit:

$100,000 = $50,000 x 2 owners

Salary:

$100,000 = $25,000 x 4 employees

Expenses:

$50,000 rent on 2400 square feet at $20 per square foot in small town USA
$25,000 bank loan payment
$25,000 advertising, credit card fees, vehicles, utilities etc

Do you see what I see?

rent is twice as high as any employee’s salary.
the bank loan is higher than any employee’s salary.

- landlord/bankers now getting 3/4 of what all employees receive combined

advertising and other expenses are also higher than any employee’s salary.

- bank just got another 3% cut (of gross!) for credit card fees

for the cost of a single year of employee income, an owner gets 50% ownership.

And maybe something you haven’t considered:

If employees work on a 25% commission, that means they earn $3 for the business for every dollar they are paid. That means the 4 employees earned $300,000 to cover the main operating costs of inventory, breakage, theft and other expenses that I didn’t explicitly cover.

In a single year, the employees have already contributed over 6 times the capital investment that the employers put in in the first place.

Yet they receive no ownership.

There are more problems with the authoritarian hierarchical structure and brutality of working 2000 hours a year in a situation where most of your civil liberties like free speech go out the window that I don’t even need to get into. I’m only talking dollars, and it’s already hopeless.

This is an intolerable situation for hackers and entrepreneurs who could very easily start their own businesses. Ones with no overhead for brick and mortar, bank loans or even advertising.

And what do you know, that’s exactly what they are doing. We’re seeing a widespread rejection of the “workplace” as industrialization gives way to automation and an information economy. Unemployment is high for a reason.

There is a land rush happening during this second tech bubble. Only the property is virtual and the freedom is real.

The people left behind in jobs are like the huddled masses of the old country, the ones trapped in cities, unwittingly subjecting themselves to a menial life of servitude, never glimpsing the frontier or tasting its freedom.

But it breaks my heart that the rugged frontiersmen of today are every bit as ruthless as a century ago. Sergey Brin is worth $17 billion of Google’s $185 billion market cap. If you split that up over the 20,000 employees, they’d each receive almost a million dollars each. He and other owners cost their employees a fortune. He has more in common with robber barons like John Rockefeller than entrepreneurs like you or I. I’m not trying to be hard on him specifically. He has some good qualities. But I think perhaps his priorities have changed. Same with Mark Zuckerberg. I think that once hackers reaches a certain level of largess, their days of positive contribution are over and they become freeloaders.

Look around at the successful tech giants of today. Do you see your value system represented? If you had a million dollars, what uses would you put it towards? I invent things, so I have a hundred world-changing ideas that Google will probably never do. I’m not sure if their 20% time would be enough for me to manifest them. I need a team to work with, I need resources. A million dollars would go a long way. Employment? Not so much.

***

I have a different vision of the future, a decade or two from now. I like to think my system is fairer. Risk, time and money are phantoms. For me, meaning is everything. Self actualization. Helping others. Triumph. Let’s paint a picture of your average mom and pop shop, small town USA, in the year 2025:

Prefab structure built at 1/2 today’s prices out of solar-kiln baked earth and recycled materials, owned not rented, 0% interest loan paid to the city instead of the bank.

- $25,000-$35,000 savings

Solar power and hydroponic garden run by roomba bots on the roof, utilities and food provided for free.
Solar electric vehicles, especially bikes and mopeds, costing under $1000, providing practically free transport.
Employees living a few blocks from work or even above/below work because they don’t hate their jobs.
Free daycare, preschool a few blocks away like in Europe, community pool, rec centers etc.
Electronic cash system, transaction fees made pointless by free fraud prevention system backed by the FBI (which it already is. what do your credit card fees pay for really?)

- $25,000 savings

Employee-owned, unionized business, equity handed out by contribution, not money or time. 6 employees paying a $250,000 loan is just $42,000 per employee, which at 25% commission takes just half a year to raise (75% of the $100/hr the service industry charges is just over 1000 hours).
Remember that fully vested means fully free. you own your stake in the business. no more boss. one person, one vote.
Successful coworkers contributing ideas that save or earn the company tens of thousands of dollars per year vest immediately.
Since money is no longer such an important objective, vested employees often opt to vest other employees to give them a shot to work on their world-changing ideas at their full potential.
The $300,000 margin quickly grows to $500,000, millions of dollars and beyond, eliminating the need for outside investors who contribute nothing besides money. venture capitalists hired as consultants instead.
since money flows like rain and the consumable needs of today are mostly met, money begins to seem less valuable (see: Finland).
no more retirement. fully vested by the first year of work also means residual income for life. why quit doing what you love when you have all the money you need? prospering research into curing cancer, traveling to the reaches of the solar system, passing on wisdom to the young and postponing death.

What I’m amazed that people fail to see is that our current economic model is the way it is for a reason. It’s exclusive. A handful of people sit on top while the rest of us work, by design. It’s as unfair as it can possibly be to the vast majority of us. In the same way that it’s perceived to be fair by the people on top.

The average employee is looking at a lifetime of work - 30 years or more, to buy into ownership of a business that the owners put just one year’s wages into. It’s so incredible to me that nobody sees this. Or does but does nothing about it.

Which is why I don’t work for businesses anymore. Working with people on the other hand - now that’s a whole different story.

Discuss on Hacker News

Sopa on a Ropa

Wed, 18 Jan 2012 14:08:00 -0500

It’s been about a month since my last post because I was pre-protesting SOPA. Ok actually I’ve been knee deep in code, doing my best to transition to a new life beyond the soul-sucking job. There’s so much I want to say about that adventure over the last year, and I will, but not today. Instead I want to talk about people that tell other people what to do.

I’ve come to realize that the systems of control we live under are mostly self-imposed. That’s a controversial statement so let me explain. When we think of the rich and powerful, we might imagine that they have resources that they can recruit to inflict harm upon us if we disobey them. That’s true if we are just looking at punishment. If you don’t pay your rent, your landlord will call the cops and they will kick you out of your apartment and you might find yourself in small claims court (but probably not, because the opportunity cost of hassling with you is higher than just finding another tenant, but I digress).

Myopic legislation like SOPA and PIPA is trying to do the same thing. You share a video online, they come kick you out of your website or whatever. Fine, we get it.

But I think this is only part of the story. The real threat of force that’s pervading the current political climate and protests around the world, in everything from the 99%/Occupy movements to the meltdown of the Euro and disenfranchisement/austerity measures of less capitalism-driven countries, is not really so much about violence at all. It’s about withholding opportunity.

If you don’t do what we say, we won’t let you play the game.

This isn’t just videos on youtube. This is about free speech and the pursuit of happiness. The core values that started America are at odds with the goals of those in power. That means that their power is phantom power. It only exists because we believe they have it. It doesn’t mean we’ll fight and die for them. Stop playing their game and the power goes away. Or more specifically, stop desiring the “opportunity” they claim to extend and they become more mooch than provider.

So much of what we think of as necessity turns out to be liability. Credit cards. Cars. Oil. Capitalism. Entertainment. These things used to be fun but now they take all of our time and effort. Lo and behold, those in power mete out these things. What we want is what makes them rich.

But there is this whole other world out there. When you step back from the rat race and remember what used to drive you as a child, your life purpose, the goal you strive to achieve, it should be obvious that these worldly desires are at odds with that. I’m not saying we should all give up our possessions and live like monks. But I am saying that instead of buying CDs and movies, maybe we should go to local shows and support independent film. Maybe we should ride our bikes instead of driving.

Maybe we should quit working for corporations. But we may lose our homes, cars or health insurance. Our children may not have the money for college. We all have to decide for ourselves how far to push the envelope in pursuit of our dreams, but we should make an informed decision, not one based on a fear of losing whatever comfort we imagine ourselves to have. Because that fear is what props up phantom wealth and power. It’s what prevents us from seeing the possibilities that open up when we stop playing into a sick system.

Maybe if we push back against the fear, we’ll start making progress as a people again. Instead of competing for a small piece of the pie, maybe we’ll work together to create more wealth for everyone. Maybe someday housing, transportation, health care and college will largely be free like in many parts of Europe and the rest of the world. I’m not making this up. Ask someone who’s been there about Finland or Costa Rica. There are other ways of being.

One of my favorite quotes is “for every step you take towards your dreams, the universe takes ten”. I’ve heard that when people achieve a certain level of wealth, they start to worry about things like their health. When the money is no longer important, they see what really is. I say, why not start now. Realize right now, this moment, that if we don’t live with dignity for a higher purpose, then we are slaves to phantom authority. People and organizations that have no better understanding of life than you do, but who choose for you, feeling entitled to, because of numbers in a bank account or a misguided understanding of their own purpose, perpetuated by the same fear and desire that ensnared us.

When POOPA or whatever they call the next censorship bill passes…nothing will happen. First off because it won’t be enforced, because legislating morality at that level is impossible.

But say they try to enforce it. Say they start doing the unthinkable, taking away people’s internet (the future’s equivalent of speech, liberty and happiness). 15 minutes after that, hackers will release new tools that create a kind of darknet we haven’t seen yet. Imagine everything encrypted. Everything onion routed. Everything anonymous. $5 wireless boxes that network neighbors, blocks, cities and eventually countries, the world and space automatically, at full bandwidth, and with automatic configuration. For free. This is a fairly easy thing to achieve, certainly much easier than maintaining linux kernels or writing the next 3D game engine. Just extend hashing networks like bittorrent to all traffic. There are millions of hackers out there, much smarter than any politician or lobbyist, and only a finite number of steps to create something like this. Some part of me dreams of it, some part of me fears it.

This is the inevitable conclusion of the intellectual property arms race. If we ever get to mutual annihilation, then communication channels will be mathematically unstoppable, so they only thing left to do will be to take away communication. They’ll have to make the internet illegal somehow, because users will be inadvertently forwarding fragments of illegal files just by using it. But they can’t make speech illegal without full authoritarianism, meaning the end of society. It’s a paradox. We’re perhaps a decade out from the arrival of the darknet, but I often wonder what will happen after that.

In fact, the more I think about it, we’re only a decade away from a lot of things that threaten power. People living off the grid. Inflatable plastic greenhouses growing practically free hydroponic food tended to by roomba-style robots. Virtual assistants and automatons that can work harder than people at any task forever. Solar powered trikes with unpuncturable tires that store their power in ultracapacitors and last forever with little to no maintenance. Clothing printed at home from polymer that costs $20 a gallon. Even the silly easy ideas now are compelling. And inevitable, on a long enough timescale.

So I’m starting to digress again and will try to keep this post digestible. The growing pains we’re seeing as the 1% tries to hold onto its remaining influence were inevitable and predicted. At some point in the near future, we’ll have a choice to make. Do we continue the class war with them, the inevitable outcome being that they become vilified, losing their wealth to pay for the crumbling society they’ve helped destroy… Or: do we stop our obsession with punishment and see beyond to what’s possible with cooperation, creating real prosperity that eclipses their phantom wealth, and with it, memories of scarcity-based limits on liberty like SOPA?

Blowback

Mon, 19 Dec 2011 22:29:00 -0500

This is two posts in one, since I was scrambling to finish a small contract and just got done.

First off, if you can’t get GLSL shaders to work in OpenGL ES2, they never tell you that you have to have BOTH a GL_FRAGMENT_SHADER and a GL_VERTEX_SHADER attached to your shader program, or else it will compile fine but sit forever and never link, and glGetShaderInfoLog() and glGetProgramInfoLog() will just return empty strings. Also you can’t pass an empty string to glShaderSource(). You will get an error in glGetShaderInfoLog() that no code could be found to parse. Pass empty main() functions instead. This shows how to attach the two shaders to your program properly:

http://stackoverflow.com/questions/4850025/shader-for-android-opengl-es

Also watch out for unicode in your shaders, it may cause problems. Use ASCII.

Second off, I was so deep in code that I put off going to the store, eating, and changing the furnace filters. Now my $1500 contract has become a $4000 furnace and water heater replacement. They were 20 years old so we did them both to save money.

Moral of the story is, you will have to go into the zone to succeed in life, because tons of things waste so much of your time that you won’t succeed otherwise. But keep your eye on the ball, follow your instincts and change the friggin furnace filters, go to the dentist, and change your oil. Otherwise it’s one step forward and two steps back.

Update: they changed the furnace today and turns out the heat exchanger was cracked so had been leaking carbon dioxide into the house for who knows how long. Periodic inspections are a good idea too.

And You Thought c++ Was Bad

Sat, 12 Nov 2011 16:39:00 -0500

UPDATE: this has been solved, thanks to Anonymous posting this link in the comments. It turns out it was very much like what I was saying, it was checking a reference to the variable instead of its contents JUST like c++ does. This is a case of there being a “proper” way to use the language, but many of the edge cases work fine so the web is swamped with examples that work through a loophole. I’m not really sure how to address this suckiness, because I will now be using “contents of” everywhere, which just made the language a bit more verbose for me. I think that Applescript has simply not lived up to the elegance that HyperTalk had, and that’s why I’m so hard on it. I want it to be good so bad :-( Oh well, thanks to everyone for your feedback and verifications.

If you are using a Mac, try the following script in AppleScript Editor:

set theItems to {"hello"}

repeat with i from 1 to count of theItems
	display dialog item i of theItems is "hello" -- true
	--display dialog class of item i of theItems -- ctxt
end repeat

repeat with theItem in theItems
	display dialog theItem is "hello" -- FALSE! #$@!
	--display dialog theItem is equal to "hello" -- still false!
	--display dialog theItem as string is "hello" -- this works...
	--display dialog class of theItem -- ctxt, which is correct?!
end repeat

I lost over an hour to this bug today. I was getting the list of window names from a program and then trying to activate the window by iterating over the list and finding the one with the same name. I could make it work by hand-entering the string but not with the one in the list.

IMHO this bug is very serious because it indicates that under the hood, Applescript has some very serious problems. I think that this might be failing because Lion no longer runs PowerPC code, and Applescript was PowerPC for a very long time, so they probably recompiled Applescript as Intel and some new bugs were introduced. So it’s probably trying to use a reference to the item in the comparison, instead of the item itself, so the comparison fails.

Imagine the thousands of work hours lost globally to bugs like this. If you are registered on Apple’s developer site, please consider submitting a bug with me.

Thanks for your help!

Who Built the Pyramids

Sat, 05 Nov 2011 15:56:00 -0400

I spent the last two posts talking about what’s going to happen because of the 99% movement and how it will affect us. We are already seeing people pulling their money out of banks and general strikes are just around the corner. This won’t have as much of an impact as people think, because the banks no longer hold money, they hold debt. So the next step will probably be people stopping their credit card payments, or forming some kind of debt union to dictate the terms of repayment. The mainstream media thinks the protests will break up before this happens, because they won’t survive the weather. But the reality is that the protestors ARE survivors. They may go underground from time to time but they endure. They are a million copies of Cody Lundin saying THIS IS WHAT WE DO, MAN.

So now that that’s settled, I want to daydream a bit about what’s driving us in the first place. I don’t know if you’ve noticed, but life is exciting again. I haven’t felt this inspired since the 90s. And it’s doubly sweet because the Nothings (the 2000s) were so profoundly bad. I’ll never go back, and my guess is, neither will you. This song sums it up nicely for me:

http://www.youtube.com/watch?v=xjoA4nYBD5U

I spent the last decade doing work that sounded ok on the face of things but left me bereft of my spirit. It’s like I was working towards the past. I knew in my heart that my dreams were dying, and it made me sound like some kind of crackpot idealist. But most everything I’ve ever said is either going to happen or is here now. So I’m a daydreamer. I take that as a compliment.

You want to know what I dream about? Stuff like this:

I have hundreds, perhaps thousands of links like this that I have stumbled upon and sent around to friends over the years.

Lately I’ve been playing Rocksmith, and after two days I can already play some songs. I barely knew what a bar chord was. We are entering an era where we can learn new skills orders of magnitude faster than we used to. Just watch, music in 2-3 years is going to be incredible, because a whole generation will be playing instruments that wouldn’t have otherwise. I don’t know how I missed this coming. I’m being surprised again - and it’s wonderful.

When we daydreamed in high school in the mid 90s, we were perhaps 10 to 20 years ahead of the curve. We wondered what the Matrix was. We talked about a post-fossil fuel world and curing cancer and AIDS. We imagined a future of people interconnected wirelessly, long before the internet and cell phones were ubiquitous.

By 2000 we were perhaps 5 years ahead of things. I remember wondering about what would come after Napster. Now we have BitTorrent and Facebook and Twitter and ways of communicating and sharing that go far beyond what we imagined. By 2005 we were less than 2 years ahead of things. We didn’t really see things like the iPad coming, at least not as a successful, ubiquitous product that grandparents buy. Today I often feel like I’m 2 weeks ahead of the curve at best. Just talking about something seems to spark it into existence. Half my searches result in me finding that something has already been invented.

That’s why I’ve begun to look further. What is coming a decade from now? Do I have time to invent it? If not, who will? And how will that change our lives?

I realized the other day, that’s what the future is. It’s the manifestation of ideas. On Star Trek, you tell the replicator to make something and it just does it. Well, that’s just around the corner. In 2020, instead of spending $30 on printer ink, we’ll spend $30 on a gallon of polymer for our makerbots to build us, I dunno, flower pots. That’s just the way it will be. Corporations that maintain the status quo, enforcing things like software patents and lawsuits against municipal wifi, just seem to get in the way of real progress.

Connected to this is ever-growing productivity, which is intimately tied to inflation and goes up about 3% per year, like compound interest. We are getting ever better at doing more with less. You get constantly better at your job, but the last time the middle class got a raise was probably 1999. Look in any industry and you’ll see individual employees doing the work that would have taken 2.5 people 30 years ago, before computers and email and overnight global shipping. The minimum wage today should be $16.50, and it actually is in most other developed countries (like Australia). People who want to lower the minimum wage to create jobs evidently want to create the kind that people do in China.

I was watching “What a Way to Go” on either FSTV or LinkTV but they said that Americans have 70 energy slaves working for them in the form of fossil fuel. Imagine how much of our prosperity (food/clothing/transportation/housing) comes so cheaply compared to the rest of the world. We can go to the store and buy a loaf of bread for $2 that a person in Africa would have to work all day for, because they have no energy slaves. That’s why in America, physical labor doesn’t pay much. A semi truck doesn’t care if the driver pedals.

Another way of saying that is that we have a 70x multiplier on our labor. It’s more profitable to make a sale than make a computer. That’s why I believe so strongly in ideas like free food and public transportation. Because it’s simply a waste of everyone’s time to work for a dollar here and there when machines already do the work. We should be aiming higher as a culture. I’m tired of rich men in suits going on talk shows and saying that we each need to pull our weight. The truth of the matter is, they are at the top of the pyramid, so if they have a dozen employees, each with 70 energy slaves, it’s like they are enjoying the comforts that a thousand slaves provide. They are living the lifestyle previously known only to lords and pharaohs. Of course they want us to pull our weight.

I saw another show last night called “What the Ancients Knew” that showed the quarries where workers pulled the stone for the pyramids. They did everything by hand with primitive tools. It turns out that the stones get smaller towards the top of a pyramid. They came in inconsistent sizes and were fit tightly into place by master stonemasons to form a perfectly straight, smooth surface.

That’s when I realized how the pyramids were built.

You want to know how? Try to build one. The secret to the pyramids is human ingenuity. You give a few thousand workers bonuses and incentives for who moves the most stone, and they will very quickly discover the best way to do it. We don’t know how they did it, because we’ve never built a pyramid by hand.

Today with our 70 energy slaves and hundreds of years of 3% productivity improvements and 300 million people, compared to ancient Egypt’s 5 million people, we could build a pyramid 4096 times bigger than they could. The great pyramid was 480 feet tall, with a base of 756 feet and a volume of 91 million cubic feet. Ours would be 7680 feet tall (1.5 miles!) with a 12,096 foot (2.5 mile) base and a volume of 375 billion cubic feet.

And we could probably build bigger than that. TNT in machine-drilled holes cracks stone much more easily than swollen wood wedges in hand-drilled holes. Train cars carrying a stone can be pushed by a single person, verses teams of dozens or hundreds of people rolling stones on wooden logs. We would find ways to raise stones with little effort using solar or wind power. We might even build our pyramid out of steel so it’s hollow inside. Future generations would never be able to figure out how we built it without fusion and artificially intelligent robots to do the work for us. They’ll have 700 or 7000 energy slaves, and virtual assistants orders of magnitude more productive than us.

So when I look around at the problems facing the world today, like hunger and low pay, I say “bah! fools!” These are ancient problems. Human ingenuity can overcome them trivially, but we don’t seem to try. We do everything the hard way, by hand. We think small. Or more specifically, our leaders think small.

We’re told that this small way of thinking is the responsible way to be, but we know in our hearts that it ignores the bigger questions about meaning and human potential. We’ll work, and we’ll do good work. But our dreams will take us further. They will provide us with bigger levers to multiply our efforts. They will take us to the stars and deeper into our own minds, and sooner than we think. I’m reminded of the Sublime song lyric: “hard work good, and hard work fine, but first take care of head.”

PEOPLE are the ones who built the pyramids. People like you and me. And they did it with far less than we have. What would YOU like to do with your life, if it weren’t for the small-minded people, institutions and policies holding you back? What kind of pyramid would you build besides the financial one propping up the overprivileged? I think that’s where we can look to get beyond the barriers holding back the 99%. Picture where we want to be. Then work backwards. Don’t listen to the ones who stand to benefit from your marginalization. Use the resources at your disposal for something great.

Watershed Moments

Fri, 28 Oct 2011 14:20:00 -0400

Yesterday I wrote about what I think is going to happen as a result of the 99% Occupy Wall Street movement. Today I’m going to imagine what it will mean for us. I’ve been curiously amused by media coverage of the protests (where there is any) because it so comically misses the point of all of this. It’s like the story of the blind men and the elephant. They each feel a different part of it, so one thinks it’s a pillar after feeling the leg, one thinks it’s a rope after touching the tail, one thinks it’s a branch after holding the trunk, and so on. They all completely miss the fact that there’s an elephant in the living room.

My life history has been based around a narrative of failed potential. If I feel anything, it’s probably the rear of the elephant, or more likely the point of its tusk or the underside of its foot. The crazy thing for me to wrap my mind around is that I got down from atop the elephant and then laid in front of it and didn’t move out of the way as it walked over the top of me. I fully accept the fact that I’m in control of my life and brought much of my misery on myself. But I will never forgot the nonchalance of the elephant as it applied stronger and stronger pressure, eventually doing a handstand on a single foot centered on my groin. And when I look around at the 400 human pyramids struggling as hard as they can to each support an elephant, I can’t help but wonder if perhaps they are beginning to realize how strange it is that we do this.

Regardless of our life experience and political persuasion, we all see the same thing. America keeps the top secure. However, the rest of us should get a job. We need to grin and bear it and stop complaining. We should try harder and stop slacking. We shouldn’t speak up when something’s wrong in the workplace or we might lose our job. We need to focus on paying our health insurance in case something bad happens. We have to save as much money as we can so we can afford to send our kids to college. Politicians have our best interests at heart as long as they are part of our political party. We can succeed if we just have faith.

If that paragraph sounded sarcastic to you, congratulations, you’re probably a progressive like me. If it sounded at all serious, then you are probably conservative. But we share the burden, even if we don’t see an elephant. We’ve both spent the last 30 years seeing no real increase in our middle class incomes even though the country has gotten an order of magnitude more efficient and productive. We’ve watched health care costs on the aging plunge them into poverty. We’ve watched public university costs skyrocket to the level once held by private colleges. The elephant was always there, and more and more of us have opened our eyes to it and jumped out of its way.

So you see, a lot is at stake here, because this is a battle of perceptions and ideas. We’ve become our own slave masters. Our chains are our sense of duty and obligation, held by the ultra rich job creators who want us to think we have nowhere else to go. If even a few people at the bottom of the pyramid decide to no longer participate, the pyramid crumbles.

When I see things like 100% of senate republicans rejecting Obama’s jobs plan because it raises taxes by half a percent after the first million dollars, this is the reason why. It isn’t about logic or what’s right and wrong or anything like that. It’s because the Tower of Babel we’ve built in America is so high now that it has a long way to fall. Tax rates on millionaires and billionaires were twice as high between the Great Depression and Reagan’s election as they are today. That means we could pass a jobs bill like this over 100 times and still not reach the tax rates we had during the peak from 1944 to 1962 (some of the most prosperous years in our nation’s history).

The 1% don’t want to compromise at all, because if they do, it’s a slippery slope to losing the position of power they’ve held for 30 years. They’re afraid we’ll find it’s easy to relinquish them of their power because we’re the ones that lent it to them in the first place. But I think we’ve reached a tipping point where the country’s health has deteriorated so much that it can no longer sustain itself in the way it has up till now. We can’t even pay the interest on our debts, much less the principal. We can’t afford to take care of or educate ourselves.

It’s like the pyramid is not just holding up elephants, it’s also being inundated by water. Even if we wanted to hold them up, we’ve begun to drown. It’s almost amusing to think of all these elephants having to soon swim around with the rest of us, but you have to consider that from their point of view, they brought this on themselves, in much the same way we brought on our servitude. Their nature is not to admit it, but to spray water on us until they find a new pyramid, and that’s ok. Our job is to buck them off our backs.

Imagine the country has been locked in a kind of primordial swamp for 30 years. No matter how hard we tried, we couldn’t swim through the muck so we got stuck. It never seemed possible that the swamp might eventually overflow its boundaries and drain someday through a crack in the shore. We always knew that something beautiful was hiding beneath the waters but we thought we’d never see it.

Well, we’re going to see it soon, and it’s going to be wonderful. Could you imagine reading a post like this during the Bush years, or under Clinton, or Bush Sr. or Reagan? No, it would have seemed ridiculous, like bell bottoms and tie die. But today you can feel it in the air. It’s on the tip of everyone’s tongue and dancing along our fingers. Something has changed in our thinking, as a culture. It’s no longer acceptable for politicians to take bribes from corporations and hang the rest of us out to dry, so they have to go. People from all walks of life and political persuasions agree on this.

It’s extraordinary that the most patriotic thing you can do right now is be yourself and fight for what’s right in your heart as an individual. It’s never been easier to be part of organizations that effect real change in the world, and they’ve never been more receptive to the really big ideas, the game changers.

With some of the smallest early barriers out of the way, like ending corporate campaign financing so normal folks like you and me can get elected, imagine how that would change our democracy. If our politicians weren’t spending every waking moment raising money, imagine the things they could accomplish as our elected representatives, like they used to do in our grandparents’ day. It will mean an end to gridlock. There is a cascade of change coming, starting with something very basic like ending corporate personhood. The thousand changes beyond that will reveal themselves once the water recedes.

This is a watershed moment in our history.

Continued here…

The New American Patriots

Thu, 27 Oct 2011 12:31:00 -0400

The Occupy Wall Street protests are finally here and it only took 30 years. I’m going to cut to the chase. I’ve known something was terribly wrong in America for pretty much my entire life. I know that you are a smart person so you did too. So let’s skip the review and do what we do best and extrapolate what’s going to happen now.

My constituency is small. I have no money (as in, less than no money, aka debt). I have no prospects, because future employers frankly can’t see as far as I do. Girls generally don’t like me, because my vision of the future doesn’t match theirs, although luckily my girlfriend is giving it the old college try. The software I use is crap. The devices I rely on are also crap, because I can design better ones myself, so most of the tasks I perform each day are a waste of my time. Politics is crap. Money is crap. I’ve spent my life working to buy food and pay the rent instead of getting real work done. I realized far too young that my body of work is pathetic, like a person on their deathbed looking back on his or her life with regret. If I were a business, I’d be so far in the red that even I wouldn’t invest in myself.

So that got me thinking. Why have others supported me? Why was I lucky enough to have nurturing parents, a good college education and banks willing to loan me money during the housing boom? I really need to figure this out, because I’m having some serious cognitive dissonance between my impression of the world and how it actually works. What do I really have to offer?

If I look at all the people I admire, like Steve Jobs, Franklin Roosevelt, John Lennon to Thom Hartmann, Van Jones and frickin Lady Gaga, I find myself wondering just how did they do it? How did they build a constituency? How were they able to achieve self expression, today’s version of freedom? Because to me, the only difference between you and me and them is the smallest sliver of opportunity. Or faith. Or affirmation. Or maybe enterprise. I just can’t quite put my finger on it. Maybe divinity. Luck?

Something profound is missing from my life. It’s almost as if I’m missing. It’s like everything I’ve tried has been a Hail Mary and so far I’ve only missed. A single successful completion - just one - and I wouldn’t be here writing this right now. The odds of that are astronomically small. Maybe too small.

In fact I’m coming to realize that it would take more than one lucky outcome to fix my life. It probably requires many small moves. No, it’s more subtle than that. More like small adjustments, applied every day. In my thinking, speaking and acting.

If I think about the future I want to see, how does what I’m currently doing fit into that? The startling realization I’m faced with is that nothing I do has any bearing on the world at large whatsoever. It’s what YOU do that really matters. Or more precisely what WE do, because I have no say in your business. I can only suggest.

So if you are still reading this, maybe you are finding yourself in the same boat I am. No money. No prospects. No constituency. So what do WE have to offer?

Well, what are we left with? Our work maybe. Our minds. Our friends. So those are our bargaining chips, however humble. As far as I can tell, if things keep going like they’re going, then we will be forced to stop providing our services for pennies on the dollar. People have stopped paying their credit cards. They are going on strike. They have no income, so aren’t paying taxes. We are beginning to unplug.

Wall street has forgotten that they are still a business and governed by the rules of the free market. Their primary profit avenue is convincing people to pay, just like a car salesman’s profit avenue is selling a car for more than its purchase price. The only thing keeping them in business is our naiveté. We keep paying our credit cards. We keep providing our services to the highest bidder, not the one most in need. We pay a higher tax rate than millionaires. It’s remarkable, in a way.

Because if we just quit the jobs we hated, and got together with our friends to form new businesses, we could trivially solve the problems plaguing the world. If you made as much in 2 hours working for yourself as you did in an entire day working for some rich guy, how hard would it be to spend the rest of your time growing a garden and riding your bike and sticking solar panels on your roof or going fishing or travelling (or doing whatever else it is you want to do)? Instead of spending money on advertising and health insurance, what if you just made a good product and brought back tariffs so you don’t have to compete with third world workers? Then you slept 8 hours a night and exercised, and invested the rest of the money in curing diseases and helping those less fortunate so they could afford to buy your products?

If everyone in your apartment got together and told the landlord that you will all be paying half the rent because real estate prices have fallen by half, would he or she evict you? What if the whole block does it? The whole city? The whole country? The landlords would tell the banks that they only have half the money now and the banks can either take it or lose everything. A natural readjustment in the free market would occur.

The problem with these ideas is that they are all or nothing. We aren’t likely to convince everyone to do them. So I think part of what the protests are doing is creating a stew of ideas, because the old ideas have been ineffectual. We’ll likely see some new constructs flowing from it that form a bridge from where we are to where we want to be.

For example, eviction insurance. If it’s too hard to get everyone to join a tenant union, then people can instead form a smaller coalition where everyone pays their rent into a central pot, and it pays out rents to the landlord. Once a critical mass of people have joined the tenant union (perhaps 10% of a city’s population or whatever the financial viability point is for the real estate market), then the tenant union would begin to ratchet down the price of rent. Once the other tenants find out that rent has been lowered some percentage, then they can also pay less without fear of eviction. In effect, they become virtual members of the tenant union by enjoying the protections it provides. Every problem our country faces has a similar, viable solution between all and nothing.

So you see, it’s not that the protestors lack a cohesive set of demands right now, but that they are discovering how to move beyond the rules set by those in power. They are forming their own constituency. Our constituency. We will soon enjoy the benefits of their union even if it’s only virtually. I’ve heard that the protestors are forming small groups that will elect representatives to meet and represent us at a national level. This is true democracy unfolding right before our eyes, and we are the new American patriots. The best thing that you can do right now is to think of a minimum viable path to move beyond your problem and then throw that idea out into the world. Also be open to the small opportunities that will come your way as others imagine viable avenues for you to move with them beyond your shared predicament. In other words, be a part of things.

Continued here…

Fix Your DVD Drive for Free

Tue, 25 Oct 2011 13:21:00 -0400

My last few posts have been kind of heavy and I decided it was time for something completely different.

So right now I’m in the do-it-yourself computer repair biz bootstrapping my way towards being a successful indie app developer. I fix a lot of older G5 iMacs and PowerBooks and the occasional MacBook. So far the most failure prone devices I’ve found are the slim and super slim SuperDrive. They fail within just a couple of years usually, whether you clean them or don’t even use them at all. I think I’ve figured out why.

My drive wouldn’t spin up when I inserted disks. If your drive is just spitting out disks with an error halfway through burning, then it most likely needs a DVD lens cleaner, and if that doesn’t work, then you may have to buy a new drive. My fix is mainly for drives that make funny sounds or fail to see disks at all. It should also work for Combo drives. I haven’t tried it on a full size desktop drive yet though.

Pictured above is the inside of a slim SuperDrive from an old PowerBook.

Before going any further, be warned that you should NEVER look inside ANY optical drive when it’s on, because they have powerful lasers that can blind you instantly. If you try the mod I did, only do it while the drive is off and disconnected, then put the cover back on for each test. Also be warned that if the drive is under any kind of manufacturer warranty, then opening the cover will void it.

I tried lubricating the central motor with some light oil and that did get it to start spinning. But it would only spin up a little bit, make some whirring sounds and then eject the disk.

I finally realized that the disk was rubbing too hard on the foam brake in the red circle. So the first test I tried was to remove the brake altogether. That caused the disk to spin up to full speed and then be ejected at high RPM. So the brake is required, it’s how the drive reduces its speed accurately (otherwise the disk would keep spinning even without power).

Then I tried cutting the foam down a bit at a time until about half of it was gone, those are the remnants in the blue square. That worked better but the disk was still spinning too fast and would be rejected half the time. So I moved the brake to the left a bit until it touched the disk with just the right amount of area.

It worked!

But some disks wouldn’t seat properly on the spindle after insertion, so would either clunk around or not catch on the spindle as it began turning. It turns out that the whole center of the drive is a lever that bangs the disk against the top of the drive housing to seat the disk firmly onto the spindle. Over time, either the gripper springs on the spindle got weak or the aluminum case fatigued just enough to not quite seat the disks. Your drive may or may not have this problem. But if it does, you can lightly bend the cover inwards with your hands to bring it closer to the slap mechanism. If you bend it too far, it will prevent the disk from spinning at all. But it wasn’t too hard to get mine adjusted properly, and now the drive works like new again!

So what the heck is going on here?

I think that these drives use tiny cheap-o motors and their magnets lose their strength over the years until they can no longer overcome the brake. So the drive tries to spin up but can’t get the disk spinning fast enough for the laser to see it. It finally gives up and just spits out the disk. Then one more drive goes into the landfill and you buy another one.

One other thing to note is that name brand disks seem to burn better in the drive than off-brands. I finally figured out why. The off-brand disks are slightly thicker, perhaps because they use cheaper polycarbonate that isn’t as strong (you can feel the difference if you put the disks side by side on a table and slide your finger along the top where they meet). This causes them to rub more on the brake, and also to not seat in quite the same place on the spindle as a thin disk, and the laser seems to be sensitive to the difference. Also the thicker disks seem to vibrate more, perhaps they are heavier so slight imbalances are magnified.

But what’s REALLY going on here?

I don’t know anything about how these drives really work but I’ll take a guess. They seem to be missing two key components: 1) some kind of directional or phase controller like an H bridge to allow the motor to slow itself down and 2) a rotation rate sensor like a hall effect sensor or optical sensor or some kind of circuit (warning: PDF) to count the voltage ripples per second as the motor spins.

Without a control mechanism to slow itself down, the motor has to rely on a brake to do it. This was probably done to cut costs, because it’s much easier to make a motor that just spins faster as more power is applied, and then use something like pulse width modulation to control the speed. A controller would use more power when slowing down (H bridges also get very hot), and a brake would use more power when speeding up. So it’s probably a wash power-wise if the drive is mostly accelerating and decelerating. Of course a controller would be more efficient at constant speed, so the lack of one could help explain why DVD players wear down your laptop’s batteries inordinately when watching movies at a constant streaming rate (RPM). Lasers use only milliwatts, so the rest of the power is probably used by the CPU to decode the MPEG.

The lack of an RPM sensor is intriguing though. This made little sense to me at first, but, with new SuperDrives costing just $60 on eBay, every bit of clever engineering helps cut costs. These drives use Reed–Solomon error correction to read disks even if they are scratched. The head reads the most data at a high RPM (but not too high or signal noise defeats any gains), so the drive constantly tests the water by speeding up and slowing down in whichever direction minimizes the error coefficient from Reed-Solomon. In this way, it automatically finds the optimum RPM. It’s pretty clever, because then there is no need for a complicated algorithm to determine the proper RPM as the read head moves in and out radially like a record player arm. If I’m completely off base here and drives don’t use a method like this yet, then they probably will someday as competition removes all unnecessary cost.

I find myself simultaneously in awe of just how amazing/inexpensive technology has become, and also how brittle/throwaway it is. Consumer opinion and brand reputation seem to be fading away as price trumps everything else. The exact formula is to minimize the number of returns within the warranty period, while maximizing profit. If the part fails the day after the warranty expires and the customer buys another one, so be it. So few companies can even make this stuff profitably, that the oligopolies ensure lack of competition. A side effect of this is that to compete with the big companies at all, you have to do something completely different. The logical next step is probably solid state storage, and Apple realized this years ago when it dropped the optical drive from the MacBook Air. So while we may complain about computers, we know that some geeks in a garage somewhere will invent something revolutionary next year. If technology didn’t stink, it wouldn’t get better.

If you successfully fix your optical drive, please let us know in the comments, and if not, well, let us know anyway!

Formal Nondeterminism

Fri, 14 Oct 2011 16:23:31 -0400

This is part three of my rant on the dismal state of computing and what to do about it. My last post was pretty vague and people have complained that I’m not offering tangible solutions. They also seem to fall into two camps, people who believe that we need to think clearly and mathematically if we wish to solve the problems in computing introduced by the human mind, and those who think that humans make a mess of computing and we need more intelligent software to do our programming for us.

After twenty years of this, I fall very strongly into the second camp. I have yet to meet anyone who has really mastered computers. And even the people who know computers inside and out tend to use ever higher level tools to do their work. A natural progression might be from assembly language, to C, to LISP, to Scala, to an exploratory tool like Matlab or Mathematica. If we take this to its logical conclusion, the highest level tool is strong artificial intelligence.

And it’s important to remember that these arguments are two sides of the same coin. In order to reach automatic computing, we need a formal foundation. As far as I can tell, none is coming from the private sector. It dictates that we can either use proper programming or make money, but not both. That leaves universities and other research institutions to do the heavy lifting. Unfortunately, they live in an ivory tower that doesn’t address real world needs like conveyability and incomplete specifications.

If we ever want to make any progress, I’m beginning to realize that we need a third path. One that takes into account human failings and incorporates them into the solution. Right now we have computer scientists on one hand and corporate masters on the other, both saying that we have to do things their way. Well, there are so many other ways of doing things that I just don’t know how much more time I can waste waiting around for them to see it.

If computers had begun more as physical manifestations (like Babbage’s Difference Engine), we might be using very different technology today. What do I mean by that? Well, in the real world, most tasks are trivially easy. Everything we do, from chopping vegetables, to pushing boxes around, to mowing the lawn, require almost no computing power. Sure they use some algorithms that we don’t typically think of computers as being good at. But they are a heck of a lot easier than something like Wolfram Alpha. Humans are remarkably adept at stepping through a series of tasks, like BASIC on a Commodore 64. We don’t think of computers as being useful for that. But that remarkable ineptitude of technology is exactly what I’m complaining about. I see it all around us, in everything we do. It’s been staring us right in the face for decades and most people aren’t even aware of it.

Growing up, we had a family friend in the carpentry business who didn’t think much of computers. He couldn’t get past the concept of a cursor. To him, a cursor is someone who swears a lot, and that’s exactly what he did whenever someone spoke well of computers. That always really bothered me, because he’s such an effective human being and deep thinker, that it seemed impossible that he couldn’t bend his mind around such a simple notion. But see, that WAS the problem. He was already skilled at his craft. Why should he have to bend his will to something that has no will?

Computers as they exist today can’t solve his needs or the needs of much of the population. If computing’s foundation was in tools instead of mathematical constructs, I have a hunch that many of the show stopping problems today like concurrency would have worked themselves out, and not just because more eyes would have been staring at them. It’s because we would have been working to solve real problems. Problems that the human mind is faced with. And those are intimately tied to everything that computers stink at today. This makes me question the foundation of computing and what we are trying to accomplish.

If I had the resources, I would start at the end and work backwards. We need more computers that have access to the real world through interfaces like X10 (only not as crappy). We need off-the-shelf macro languages where you demonstrate to the computer what you are doing, and it learns the logic required to accomplish the same thing. For example, moving a drill press through a range of motion and then training the computer to do it (and eventually do it better). Or giving a spreadsheet a list of numbers and the answer and letting it come up with the desired formula. Surprisingly, solutions similar to these already exist to some degree in primitive form. I’ve always wondered why they’re not mainstream, but I have a hunch that it’s because people want to be in the driver’s seat. We’re used to thinking of computers as dumb appliances and we aren’t quite ready to hand off responsibility to them. But that saddles us with needless drudgery.

This is all rooted in control issues. By that I mean a power struggle, like between a strict parent and a rebellious teenager. Right now computers are formally described down to their lowest level. At least we pretend they are. If you use a language like C or java or an operating system like UNIX, it’s actually impossible to prove that systems will do exactly what we want in every situation, as I showed in my last two posts. We like feeling like we understand what computers are doing, and we go to great lengths to pretend that problems don’t exist. But it’s all a fantasy. I first grasped the enormity of the problem about 15 years ago in college, but now more and more people seem to be waking up.

So if we accept the fact that we aren’t in control, to what degree are we willing to give up control? For some people, a bandaid like bounds checking of arrays in high level languages, or statically typed variables is enough that they can sleep at night. For others, they aren’t happy unless code has gone through rigorous static analysis and unit tests. Still others might use probability as a cushion, like six sigma reliability in air traffic control systems.

The remarkable thing to understand is that in the end none of this really works all of the time. It all comes down to the skill of the programmer. A bad programmer will write bad code in any language. And worse, detection of malicious code is still in its infancy so we’ll likely never even know. I was just discussing the problem with my girlfriend the other night. She wants to go into forensic accounting to sniff out fraud, but there is no real analog in the computing world. The best we can do is hope that an employee is a good person. Check out the obfuscated C contest. It’s difficult to detect maliciousness in a page of code, much less thousands.

If you still don’t believe me about the significance of this, I understand. But if you bear with me for a moment, I want to extrapolate. If control is just some statistic that satisfies us at some level (99% assuredness, whatever), then it must become smaller as complexity increases. There is only so much that a single human mind can grasp at once. But computer programs grow forever, often past millions of lines of code today. More and more of what computers do will inevitably become a mystery to us, no matter how much skill we have. Unpredictability is coming whether we like it or not.

So holding onto this fantasy of control makes us feel better, but at what cost? Are we willing to disenfranchise 90% of the world’s population because we believe they should internalize the same value system we have and bend their minds around the shortcomings of today’s technology? Don’t worry too much about this, because it will never happen.

If we instead picture a future where technology has been adopted by the majority of the population, what will it look like? Computers will inevitably change somehow. It’s like we are living in the sysadmin world of the 1970s before Xerox/Apple and GUIs brought computing to the masses. Nobody then could predict that computers would be used to share cat videos on facebook. Computers are evolving now in strange and unusual ways. All of the interesting frontiers in computer science are of a distributed nature and built on the democratization of technology.

And we can’t see past those boundaries because our thinking is rooted in systems of control that are becoming ever less relevant as time goes on. All I’m saying is that instead of putting so much time, energy and money into incremental improvements, perhaps we could consider another way. One that allows for a certain level of uncertainty in exchange for orders of magnitude more freedom.

The problem as I see it is that right now tremendous research is going into making processors faster, but this is futile. By its very nature, serial computation will do little to help people in the real world where distributed solutions like machine learning are so badly needed. We are thinking in terms of gigaflops, not teraflops or petaflops and beyond. Computer scientists just don’t seem to realize how cheap transistors have gotten and how easy it is to build a chip with thousands of cores. It’s far simpler to build an FPGA than the successor to the Intel i7. So they don’t even bother to notice when trivial solutions like MapReduce solve a difficult problem like search in effectively O(1) time. I just wonder how many other “difficult” problems in artificial intelligence have simple answers when we aren’t bound by the number of processing units.

I think this myopia comes from not having a general way to split a problem up into manageable pieces, some of which can be solved in parallel, and then put them back together again. Everything stems from this, because without parallelization, we have no good divide and conquer strategies. We can’t explore the large problem spaces surrounding issues like automatic algorithm generation and verification. Which means we’ll have to write algorithms ourselves, and that means no artificial intelligence, and worse: no relief for programmers forced to deal with ever-increasing levels of complexity. Until this changes, digital assistants will never be available to the masses.

Which is why it occurred to me that out of all of the problems that I’ve encountered in programming today, parallelization bothers me the most. Because it’s so elegant. It’s like finding a hole in your life raft. Sure you can put your finger over it, but it will eventually cause you a great level of discomfort. Anyone who thinks that we can work around it by avoiding iterators or other low level patterns is in denial about the chaos that happens in large systems.

Concurrency issues and other problems in these systems just never seem to go away, no matter how hard you try to move past them. We have some models that work currently, like ACID compliant databases. But you can still build a system on top of them that deadlocks. There is definitely no silver bullet here, perhaps even more so than at the language level. That’s because large concurrent systems are nondeterministic. Humans are simply unable to design them to be completely safe and predictable. And what’s a larger concurrent system than an artificial intelligence?

With so much riding on this, and after 50 years of research into parallelizing functions, has there been progress? Is there any hope? These questions need to be articulated and known solutions need to be summarized. Because right now it takes so many years of study to reach the point where we can even grasp these notions, that we can no longer see the forest for the trees.

What we really need is formal nondeterminism. We are living in a Newtonian age of classical computing but we’re moving into a kind of quantum era of probabilistic problem solving.

To get a glimpse of what I’m talking about, check out this article on ATNs, it’s very approachable and shows how a first-effort natural language parser works. Now imagine if the tree is enormous with millions of connections, and every path down each split in the tree needs to be executed in parallel if you have any hope of doing it in a reasonable amount of time. If you tell the parser to come up with its best guess in a limited timespan, then it just became nondeterministic because one branch might finish before another on subsequent runs. So there is a chance of getting a different answer each time, even from the same sentence. This is a remarkable result, and happens to humans all of the time when we are under pressure to solve a problem quickly.

So with AI, these probabilities are constantly coming up. With something like Apple’s Siri, it will be immediately obvious to every user that the computer doesn’t always interpret things correctly. But neither do we, if the room is noisy or we are distracted thinking about other things. And that’s ok. And as computing power increases, the computer has more time to work on a problem and so becomes more deterministic, at least for known sentences and contexts.

But the world is more random than that. Every situation is different. So a real AI will never really have complete certainty about much of anything, just like us. Everything will be shades of gray, which means computers will be fallible even when they are functioning properly, which hasn’t been much of a problem till now.

My solution to this is to not address that problem directly, because frankly there are a lot of really smart people out there, and if they haven’t solved it by now, they probably aren’t going to. I have a hunch why that is, but can’t prove it.

The gist of it is that we can write the equations for a quantum computer to crack encryption problems, but we need an actual quantum computer to solve them in real life. There is no way to simulate a quantum computer, because the computer emulating it would be much too slow. We’d get the answer 100 lifetimes of the universe from now or whenever. My sense of this is that even if aliens gave us the theory for intelligence today, we couldn’t utilize it right away because we don’t have the appropriate hardware. Even all of the computers in the world with their petaflops of computing power are too isolated with too much latency to think like a human in real time.

So let’s build something almost embarassingly simpler and more interconnected, and just give it to lots of programmers. A teraflops chip with a thousand cores running at 1 GHz with Erlang would only cost a few hundred dollars in large lots and bring supercomputing to the masses. Give it simple ways of interacting with the real world and let it start solving real problems. Maybe this completely sidesteps how to build R2D2. But I have faith that once we reach a critical mass of people, their needs will lead the way.

Short of that, maybe we can build virtual machines to simulate a thousand cores running at 1 MHz and at least play around with the math. It may not do much more than image recognition but it would get a whole generation thinking about solving problems in a new light.

There has been too much emphasis on correctness and exactness when researchers should really be focussing on charting boundaries so they can be smashed. This is more like imagineering than engineering and there are multiple routes that will get us there. I would very much like to work on a project like this, in fact it’s pretty much all I think about anymore. I wish so badly that I knew more people who were interested in this stuff, with the time, money and motivation to make it happen. I just have this gut feeling that it’s not as hard of a problem as people think. Arithmetic was hard before calculators, so now all we need is a calculator for thinking, or at the very least, finding needles in haystacks, because that’s most of what thinking is.

A Sufficiently Advanced Bullet

Wed, 05 Oct 2011 15:22:00 -0400

On Monday I wrote an article about the miserable state of software engineering and it seems to have struck quite a nerve, judging by the comments and discussion on Hacker News. The gist of my argument was that designing software is very much a hands-on process right now. The computer gives you no leniency at any step of the operation, which means that it takes developers years to perfect their craft. Time that I feel could be better spent improving the conditions under which we work and the assumptions we use to frame our approach (which I will shed some positive light on later in this post).

So it’s ironic that the very next day, Apple announced the iPhone 4S with speech recognition technology that is a solid attempt to move computing away from the sort of rigid orthodoxy that’s defined it up till now. Especially since I said:

“…today’s computing can’t take us into the future. It can’t provide true artificial intelligence or bring the kind of multiplication of effort that hackers take for granted to the masses.”

Hmmm. What the heck did I mean by that? Let me explain…

I’ve been meditating on why I’m in such a conflicted state about technology. On the one hand, it’s getting faster every year and will inevitably improve by brute force. But on the other hand, it’s based on many archaic assumptions that made sense thirty years ago but are increasingly hindering our ability to move forward. It’s like it’s getting worse as it gets better. It’s taking herculean efforts now to squeeze out incremental improvements.

For example Siri, the company that developed Apple’s speech recognition technology, invested $24 million to do it, and Apple spent $200 million purchasing it. Really? $200 MILLION dollars?

Someone could hire 200 John Carmacks for a year for that. Or 4,000 run-of-the-mill developers in America. Or 20,000 in India. Even if using the $24 million development figure, the numbers are only about ten times smaller, but still staggering. It simply costs a tremendous amount of money to advance the state of the art. And the risk of failure is too high for all but the largest companies. So odds are, you won’t be doing it. And neither will I. Which makes me question why I ever got into this business in the first place.

Computer science has priced itself out of existence.

They say there is no silver bullet to fix software engineering. This is self-evident to most people working in the field, but has tons of supporting evidence in terms of how much time, money, manpower or any other resource gets thrown at programming problems with little benefit (see: mythical man month).

But it can actually be proven mathematically. The halting problem asks whether it’s possible to tell if a program and arbitrary input will finish executing (without actually running the program). This is a fancy way of asking if you can tell whether a program will ever give you an answer before you run it. Alan Turing proved that the answer is no, that the halting problem is undecidable over Turing machines when given an arbitrarily complex program. And since computers can be emulated on Turing machines, that means that no computer scientist can say with certainty whether a program will provide an answer in all cases, or if it will just lock up on him/her, without just running the dang thing.

Since writing the last post, I realized that a ramification of this is that since Lambda calculus (the basis of lisp and other functional languages) has been shown to be equivalent to Turing machines, that all of the “modern” languages like Erlang, OCaml, Haskell, Clojure and Scala are also undecidable. We can throw all kinds of static analysis at them, force them to use an explicit type system, force all data to be immutable and use every best practice we’ve ever come up with.

It will do little to alleviate the fundamental problem that we just don’t know what a program is going to do in the real world until we run it.

This is that sense of dread you feel when you first look at someone else’s code and can’t decipher it. It’s that nagging suspicion in the back of your mind that maybe you forgot something and your code might break someday. No matter how prepared you are, or how much work you put into your code, you can’t prove that it’s bulletproof in the real world. You have to run it somewhere to find out, probably in the computer of your own mind, over and over again. It’s complicated and exhausting. It’s so hard in fact that when I’m the zone, I can’t even talk to someone or my program crashes and I have to start from the beginning so that 15 minutes later I can get back where I was. I would say that of the 10% of my time left over after I work around all of the other headaches (compiler issues, administrative tasks, failures of communication), that 90% of THAT time goes to thinking about what my program is going to do. I actually write maybe a few dozen lines of code per day.

So simulation is not just a good idea, it’s the bulk of what we do. And computers today are terrible at it.

I’ve sort of come to embrace the fact that I don’t have some kind of magic bullet I can use to program my computer for me. It’s liberating to know that other people don’t have a secret formula that makes them more effective than me. But they do have better tools and methodologies. I think the best example of that right now is how Notch wrote Minecraft in java of all things. He’s able to step back and see the big picture, that anything a language does to help visualization, exploration of problems/solutions and simulation is a win.

I guess that’s why I like php. I don’t have to do a bunch of setup, I don’t have to wait for it to compile, and I can rapidly refine the results. I wonder if people would be so put off by php if I compiled scala to it and ran that. Or if I wrote php-like macros in lisp. It’s nice to think that with the right language, we avoid issues down the road. I hate to break it to you though: if it runs on unix, it’s going to have issues anyway, probably sooner than you realize. And even if it was a lisp machine all the way down, it would’t avoid the decidability problem. I’m starting to wonder if all that matters is my own leverage, using whatever tool helps me work the fastest. As someone mentioned in the Hacker News discussion, perhaps Worse is Better.

The problems we face in life are akin to ones inside computers, since life could very well be a simulation. When entrepreneurs mention iterating and pivoting faster than their competition, they are talking about simulation. Up till now, I’ve done a poor job experimenting in my own life. While I tend to daydream about other ways of living, I haven’t put enough effort into pushing the boundaries of my existence. I have a profound disappointment in my progress. That’s probably why my last post came across as cynical, because I was talking about everything, not just programming. It’s naive of me to think I can fix any part of the problem, much less the rest of it. But maybe I write because I’m hopeful that in some way, I can do exactly that, by throwing a little fuel on the fire.

The problem as I see it is that the industry has backed itself into a corner and can’t even see that the way forward requires thinking outside the box.

I’m going to get a little abstract now because what I propose is not really a well-formed concept. Arthur C. Clarke said that “any sufficiently advanced technology is indistinguishable from magic.” Well, we know there is no magic bullet. But that’s no excuse to just throw our hands in the air and give up. Maybe the problem only seems intractable. Our brains obviously work. Somehow nature iterated over billions of years and came up with us. We shouldn’t have to wait that long. We can learn from ourselves and go beyond our original programming. For all of our technical prowess and left-brain success mastering technology, it lacks feeling. There is no place for introspection in computing, and no spark, no meaning. Technology is the antithesis of zen. Maybe we should recognize technology’s limitations, come to peace with that, and just do our best. There may never be a magic bullet, but maybe with tools already at our disposal, we can design a sufficiently advanced one that it won’t matter.

I can hear the groans now: “you can’t fix technology with less technology!” or “how can we trust computers if we don’t understand how they work?!” Or something perhaps more relevant: “if computers start programming themselves, we’ll be out of a job!”

I sympathize with all of these arguments. But do we really completely understand how computers work even now? We may know some of the theory but we are fooling ourselves if we think we can watch the hardware and software and not learn something. It’s like life before having children. We can’t picture a world right now where computers are our peers, or at the very least, able to do many of the things that we do now. But once it happens, we won’t be able to picture what life was like before. That’s a frightening thought, but people have been having children for millennia. And whether we like it or not, we’re pregnant. Nine years from now (or nine decades, whichever timespan you use), we’re going to create artificial life. And it’s going to be ok.

It’s hard to imagine what form this life will take. For some reason, we have a hard time understanding how we break down a problem into a series of simpler steps. Existence is all encompassing for us. We don’t tend to separate our conscious mind from something like image recognition when we read. We forget the years we spent listening to the mutterings of people around us to learn language. We are made up of a hierarchy of networks utilizing other networks, built on yet other networks. All evolving in real time, hungry for knowledge and doing their darnedest to do a good job and be part of the conversation. But we don’t even know they exist, because we are all of them simultaneously. For a little background on what I’m talking about, take a look at this presentation by Ben Goertzel on OpenCog. It’s an open source project to simulate intelligence as hundreds of software agents working in parallel.

One of the most promising learning systems right now is probably MOSES, which from what I understand works like genetic algorithms but instead of a fixed fitness function, uses probability to direct evolution. This is part of a family of similar algorithms like simulated annealing and ant colony optimization. I have a hunch (though I can’t prove it) that we’ll eventually end up with a graph of graphs made up of hyperlinks (honestly not much different from the internet), with something akin to virtual ants walking the links at say 100 Hz initially. There will be something like neurotransmitters that feed or penalize portions of the graph depending on current conditions in the real world. You can think of it like thousands of people pushing the Like button on something interesting and triggering a flood of others to the food. There will be other processes at work to simulate cell death or damage to the network and the random formation of connections. But only a finite number of processes, each of which is easily understood. And this will all be simulated in something kind of like Apple’s Time Machine, where the current state of the network is smeared backwards and forwards in time (really along other dimensions because the connections would be based on time), with something like MOSES evolving these networks and subnetworks and rejoining them to the current state as they perform better, much like how quantum and chemical reactions let our 100 billion neurons compete and cooperate in parallel.

You can make a strong argument right now that the internet is already acting much like a living thing, except that its lowest level neuron is a human. That will begin to change as more of the functions are carried out by artificial intelligences like Siri. Eventually a few decades from now, we’ll probably see intelligence evolve from the web as an emergent phonomenon that no longer needs human interaction to function.

Which is why I feel that we don’t need to come up with an overarching theory of intelligence. In fact, by pouring research into ever more sophisticated mathematical models of learning, we may actually be hindering our progress. Because that stuff is exclusionary. So few people understand it, and so many people feel like they are on the outside looking in, that those efforts will almost certainly fail. Not to be hard on Watson, but who cares if we write a program that can play Jeopardy if that’s all it can do? The interesting problem, and the one that nobody has solved yet, is how do we write a program that can write Watson?

Imagine what an intelligence like that would be asked to do. It’s akin to learning the rules of chess by watching a series of moves. We can write programs that can do this now, but moves like castling throw a wrench into the logic and force the computer to reevaluate its assumptions. THAT is the important part. And it needs to be able to reliably do that without becoming unstable, if it’s ever going to scale up to tackling harder problems.

A true artificial intelligence will learn the same way we do, from nothing, and will be rapidly simulating and refining its notion of reality. It will have the same properties as us. It will be able to suffer damage to half its network and still function at close to its original level. It will be able to rewind in time to previous states without risk. It will be able to play out its best guess numerous times in its simulator, the same way our imagination does.

What we really need is to create a cradle of civilization for artificial life. We need to set up the initial conditions that favor its evolution. Right now sequential computers are nowhere near up to the task. They are kind of a joke actually. What we really need are systems more like FGPAs, with thousands of cores running algorithms like MapReduce, where the data is processed in-place. If computers have a hard time simulating more than a few hundred neurons, let’s just get rid of that hurdle altogether. Simulating a large number of neurons (say a million) is not a technical problem, anyone could make a chip to do it right now with a few geeks, a fab, and a million dollars.

There is a valid argument that we don’t know how to program large numbers of neurons and that intelligence has so far never evolved from a system like that. I’m going to skirt that issue for a moment though and imagine an existing processor like Intel’s i7 arranged in an intelligent way, with its billion transistors being used for say 100x100 or 10,000 cores, each with the same 3500 transistors the MOS 6502 had, plus some overhead for interconnect, with no cache memory or pipelining or any other nonsense, running at say 1 GHz because the clock speed isn’t that important. Then lets take 10 GB of ram and split it up into 10,000 pieces so each core has 1 MB of ram.

Now if it was 1980 and you gave Steve Wozniak 10,000 cores running at 1 GHz, each with more ram than 90% of computers of the time, can you imagine what he’d do with it? Would text to speech technology seem like anything special? No of course not, a child could program it. Image recognition would be a joke, any algorithm would work well with that kind of horsepower. In fact, I think most of the things we are just seeing now, like speech recognition and face recognition and handwriting recognition could have happened 20 or 30 years ago if there had been an economic incentive for them. That’s why I’m underwhelmed to say the least when I see them today. It’s one small step for man, one giant leap backwards for technology.

That’s why Apple had to make a special image signal processor for the face recognition in the iPhone 4S. That’s why today’s computing can’t take us into the future. But something of the same stuff but different arrangement will.

I need a coprocessor in my computer like this where I can run 10,000 virtual machines and simulate the programs I write so I don’t have to do it in my mind. The cores should be looking ahead and suggesting where my function will fail, like a preemptive debugger. They should be trying every possible input for the type of data I’m using. They may not be able to simulate every possible scenario (because there is no silver bullet), but they could give me a confidence level. They could be translating the functions to lisp and evolving them with genetic algorithms and then translating the corrected version back to show me a better way. The possibilities are endless, but I want to stress that they are not technically complicated. They are just resource intensive. But our processors are already sitting around wasting 99.999% of their computing power, as I explained in my last post. We have more resources than we realize, if they were only allocated in a way that we could use them.

I need to be able to run Xcode’s distributed compiling on the cores, so all programs compile instantly. I need a good communication layer like Erlang to handle parcelling work units out to the cores, my neighbors and beyond. I need very basic infrastructure changes to make this stuff possible, but I just never hear about any. The closest thing we have right now is probably CUDA running on GPUs (yet another abstraction layer). The way I see it, it’s going to be a long time before this stuff happens naturally.

But if we had it right now, we’d have all you smart people making the most of it. We’d have geeky friends bragging about their million cores. Recruiting that much horsepower will happen naturally, and very quickly. We’ll be able to observe artificial neurons tackling real-world problems and begin to develop theories about their functions. It’s much easier to recognize solutions than to construct them.

And if artificial intelligence is no closer at that point than it is now, then at least we’ll have framed the problem in a new light and can do the next big thing. Or maybe the bullet will be sufficiently advanced at that point that we won’t need to.

I believe in my heart that when we discover the algorithm for learning, it will fit on a napkin. Kids will wear the image of the basic artificial neuron on their T shirts. This stuff could literally happen before the decade is out. I don’t know about anyone else, but for me, the biggest hurdle to working on stuff like this is just making rent each month. My short game stinks.

I guess I will talk about fixing my real life simulation in my next post, but this got way too long so I’ll leave it at that.

Concluded here, at least for now!

The State of the Art is Terrible

Mon, 03 Oct 2011 06:04:00 -0400

I stumbled onto this post the other day through Hacker News. Basically Ryan Dahl (the guy who wrote node.js, a web framework that makes running web servers easy with javascript) called out the deplorable state of software today.

I fully agree with him.

And here’s why - I’ve seen things. In college I learned how to design a computer from the ground up at the mask level. We learned the equations that attempt to describe the properties of semiconductors and applied them to hundreds of problems, everything from amplifiers and logic gates to full adders and CPUs. One of my favorite projects was wiring together a primitive robot from logic gates that followed a line on the ground.

I wrote programs and games in assembly language on both the x86 and 68000 processors in the 90s. I even went the level below that and wrote opcodes in binary to interface with hardware. I learned hardware description languages like VHDL for controlling FGPAs. I’ve also dabbled in functional languages like Scheme and can tell you the ins and outs of probably a dozen different languages and methodologies from macro languages to query languages. I can tell you why D or javascript are better than c++ for instance (and no, it’s not just about prettier syntax or garbage collection).

I helped write a commercial game in the Mac App Store that briefly got to #6 in top paid. I’ve installed linux and a full LAMP server remotely. I’ve compiled and installed dozens of open source libraries from Ogg Vorbis to Freetype. I wrote what was effectively a TCP stack above UDP for our now defunct online environment. I underestimated the difficulty of network programming (so far the most difficult subject I’ve ever encountered, probably somewhere between 3D and AI in complexity). I know why atomicity, REST and ACID are fundamental and if you think you’ve found a way around them, you are on borrowed time.

I’ve worked around dozens of compiler bugs and have been knee deep in DLL hell. I’ve dealt with the needless myopia of tools that don’t understand spaces in names or case insensitivity. I’ve overcome poor planning on the part of software architects.

And you know what? After everything I’ve seen, after traveling to the deepest recesses of the Matrix, sometimes I just kind of look off into the distance wistfully and think about what might have been. It really, truly, is all crap. And it’s so much worse than anybody realizes.

When I read about the elegant computer science of the 50s and 60s, when great minds like Alan Turing and Peter Landin laid down the foundations of computation, I have to admit that I’m jealous. It’s so quaint to write lisp on a whiteboard. It’s adorable when programming language tutorials show how to print “Hello, world!” Does anybody really use that stuff anymore? No, of course not.

Software engineers today learn the quickest way to look up a code snippet on Stack Overflow and then load a PNG in iOS with a cryptic language like objective-c and then scratch their heads and wonder why the alpha is baked into the color channels. Then they find themselves knee deep in a conversation on chat/irc/a forum somewhere debating the relative merits of file size vs. ease of use. After reaching a stalemate with coworkers, they end up doing everything twice to satisfy both camps and carry around the baggage of added complexity for all time. The code grows and grows as it’s made to work cross-platform and eventually breaks one day when someone unfamiliar with the reason for the complexity decides to axe it in favor of the One True Way, which breaks some other app he or she isn’t working on.

This is a story about one line in one file. But there are a hundred files, each containing a thousand lines, in a million apps. It doesn’t matter if you learn how to do it better next time. There are infinitely many problems, so if you think it ever gets any easier or that you will be spending any less of your time overcoming these types of obstacles, you are fooling yourself. It gets worse. And worse. And even worse.

Software engineering is a misnomer. It’s more like software fabrication. As in fraud.

I think the last time I felt like I was dealing even remotely with any kind of computer science was when I used Matlab at hp back in 2005. Matlab’s primary focus is leverage. Let the code be a multiplier that amplifies a programmer’s abilities.

But then I went back to c++. Its primary focus is verbosity. Force the programmer to be explicit, no matter how small a problem is. In any given 12 hour day of programming, I’d say less than a single hour goes to writing new code now. I get up, in a haze I remember the list of bugs from the day before, I fix one and trigger three others, then I go on google and research a problem with our SVN server for four hours because it doesn’t want to check in the library file I just compiled. Rinse, repeat. Every day is some new horror, some deeper dimension of absurdity that slowly drives me into madness.

Most computers today, for all of their potential speed, are largely a mistake, based on the provenly unscalable Von Neumann architecture, controlled with one of the most shortsighted languages of all time, x86 assembly. They are almost unfathomably inefficient. Their processors have close to a billion transistors, most of which sit idle while a tiny fraction of a fraction of them perform some operation. Three quarters of a processor may be devoted to the quagmire of cache memory and its demands. All of this brute force horsepower gets stacked in an ever higher tower of babel in the relentless race to perform more sequential calculations per second. If people only know what engineering was required to implement branch prediction and 20 stage deep pipelines… It’s like seeing being the walls of a meat packing plant. You just don’t want to know.

If you knew that your computer performed two or three hundred empty cycles waiting for some piece of data to be fetched from main memory on a cache miss, or that when you see the little spinny thing, you are actually waiting for your hard drive to track down dozens of fragments of a file scattered across the hard disk drive because it got too full that one time, or that your web browser locked up on you because some novice programmer wrote some portion of it in blocking network code that is waiting for the last byte to arrive from the web server, and that the web server is sending that byte over and over again because a router is temporarily overloaded and is dropping packets like crazy so your neighbor can download a youtube clip of a cat hitting a ball into its owner’s crotch, you might throw up in your mouth a little bit. Sure, your computer can perform 10 billion floating point operations per second. But most of the time it’s not doing anything at all. Just like you.

Once again, I find myself writing another rant about things that are as obvious to me as a child’s love for its parents. But I’ve spent over 20 years immersed in this crap. Most people are blissfully unaware that computing as it exists today acts as more of a barrier to progress than an avenue.

Let me give you an example.

You might have an idea for an iPhone app, but I can tell you with all honesty that you won’t be able to write it. Because first off you won’t want to pay the $100 to be an Apple developer. Then you won’t understand how provisioning works and for the life of you, it won’t make any sense why you have to sign your app cryptographically to make it run on your own iPad. Then after spending two weeks learning objective-c, it won’t help because you won’t be able to get the game development framework you downloaded to compile on your Mac, because you also need to install SDL or a javascript interpreter or MacPorts or some other tool you aren’t familiar with. You won’t understand the unix command they tell you to use, to configure and make the library. You won’t realize that there are two compilers on your computer, gcc and LLVM. You won’t know that even if you choose gcc, that the newest 4.2 version sucks and the library only compiles with 4.0. And you won’t know how to change the project setting in Xcode, because there are also target settings that can override them. Even if you manage to make it through all of these hurdles, and the dozen more that I haven’t even mentioned yet (I’ll give you a hint: corrupted project), there is still the matter of submitting the app to Apple, which won’t work either. When you decide to add iAds to your app, they might work a few weeks after you start. But when they do, you won’t make any money. Then you’ll learn about the ad services that serve ads from multiple agencies and the idiosynchracies that are required to implement them. And after all of this, three months later, when you are earning $1 per day in ad revenue, you won’t know how to deal with that empty feeling you get when you hear how many millions of dollars Angry Birds is making. You won’t have an inkling of what people who have been in the business decades are feeling. And you won’t know that you should have been investing your time in affiliate marketing and just hired out the development to some poor shmuck who got sucked into this world before knowing what was entailed.

Because they know what you are just starting to grasp. That it shouldn’t be like this. That computers have vastly underserved their users. Conceptually, mobile and casual interaction is the future of computing. But it has no formal basis. It’s a beautiful shrine built on a foundation of tinker toys. The good tools like functional programming and provably correct algorithms are either too esoteric or too expensive for the mainstream. So far all investment has gone into racing ahead, instead of planning what type of future was being built. Today companies decide what tools you will use and the manner in which you use them, and are driven by profit, not progress.

And here’s why I find that at least a little demoralizing. Because the real secret they won’t tell you, heck, that I think is only dawning on a few people, is that today’s computing can’t take us into the future. It can’t provide true artificial intelligence or bring the kind of multiplication of effort that hackers take for granted to the masses. Computer science has utterly failed to tackle the real world problems, things like automating jobs so people don’t have to work, or working hand in hand with humans to explore solutions we have trouble seeing ourselves. We are so far from a Star Trek-style future utopia that it breaks my heart.

If you want to see the future of computing, look to every place that computing fails. The world is crying out for loose programming languages that just insert the darn semicolon instead of telling you that it’s missing, or dispense with it altogether. Sequential computing is at a standstill, and has been for close to 10 years now. We are long overdue for parallel computing. We need computers with thousands or millions of cores to better recruit the billions of transistors in chips today. We need better maths to find, describe and simulate the networks that will let these parallel computers communicate. Evolvable hardware and self modifying code need to go mainstream. Programming, at least the fundamentals, should be taught in elementary school just like any other language, but the teachers don’t yet have the required familiarity, so we should start with them. We don’t even have computers we trust enough to handle machinery, so we subject humans to a 21st century version of slavery in manual labor. I could go on and on and on about how a $700 iPad has about as much to do with revolutionizing computing as a TV hospital drama does with revolutionizing medicine. It’s all a farce, a sham. It’s giving candy to starving children.

I’m on the threshold now of rejecting this false idol, but for at least a little longer I have to cling to it to carry me through. I have a dream of starting some kind of open source movement for evolvable hardware and languages. The core philosophy would be that if your grandparents can’t use it out of the box to do something real (like do their taxes or call 911 when they fall down) then it fails. You should literally be able to tell it what you want it to do and it would do its darnedest to do a good job for you. Computers today are the opposite of that. They own you and bend you to their will. And I don’t think people fully realize how trapped we are within this aging infrastructure.

Think of the most programmable application out there, maybe something like Microsoft Excel. Do you know anybody who can actually get the macros to work? Probably not. And most programs are far worse than that, or don’t support programming at all. Visual Basic and Applescript are the laughing stocks of the programming world. For the most part, a user is either a programmer with full control (a tiny minority) or a novice who doesn’t even know that it’s possible to alter a program. In other words, the majority of users are algorithmically illiterate. That’s a travesty in 2011. Heck, my Mac Plus in 1987 with HyperCard was more approachable than anything today.

I guess I’m writing this to speak out against the status quo. I’ve heard a lot of regular folks criticize geeks for creating this mess and idolizing technology with a kind of zealotry, but what they don’t know is that many of us are just as appalled as they are. I don’t know if I can take hearing another self-aggrandizing nincompoop get up on stage and tout their amazing software. They talk so much about how the secret to their success was some tool or observation or trick. No. The truth was a combination of being in the right place at the right time, surviving all the B.S. that life threw at them when they were down and out, and frankly luck. The actual part about writing a new tool is such a tiny part of the equation that you could practically leave it out altogether and still find successful companies.

I guess I am just full of criticism and not offering a lot of solutions. So be it. I don’t expect any sort of revolutionary transition to truly innovative computing any time soon. What’s more likely to happen is that a few really smart people will write tools like node.js that encapsulate the nonsense so we don’t have to deal with it.

That’s a reason why one of my favorite languages is php. It just works. Screw up your variable typing? Who cares. Forget a symbol somewhere? Heck you still get some output and can see your error. All that matters is that for a given input, you get a certain output. In a way, php is acting more functionally than real functional languages. That’s why it’s a hacker’s language. It doesn’t get between you and what you are trying to do. I need something with the flexibility and ease of use of php, but the formalism of lisp. Matlab is about the closest thing I’ve found.

But I would take it further. I’ve often thought about writing a human language based syntax like Hypertalk that has the power of Scheme. I don’t care so much about speed or efficiency. What I really need is expressivity. The language should adapt to my needs instead of throwing its hands in the air when I told it to give me a list’s size when I should have asked for length or count.

Rather than going on at length about how I would do things differently, I will instead ask you to be open to the possibility that some of what I say is true. That we ain’t seen nothin’ yet. That the powers that be prefer to keep things the way they are and make millions of dollars a year instead of liberating us from restrictive technology. One might look at someone like Richard Stallman and think that he resembles a crazy person. He has perhaps seen too much. But his adament stance on free software is to be commended. If we don’t speak out against authority, the transnational oligopolies of the world will eventually force us into a sandboxed existence where we rent our technology from them for a recurring fee and they’ve patented every method of computing so we can’t use it. It’s scary how many aspects of our lives already work this way, and how many millions of people are completely oblivious to it. I still think this fight for the future is one of the great battles of our time. And like any other offensive, capturing hearts and minds is how it will be won.

Which is why I wrote this, another rant in a long list of many!

Continued here with a possible solution

Neutrino Antigravity?

Wed, 28 Sep 2011 13:38:05 -0400

You’ve all probably heard the news from CERN about faster than light neutrinos. Here is a good discussion about some of the ramifications of this discovery if it pans out.

I’ve stumbled onto some other articles lately that are pointing to some weird interactions with gravity:

http://www.physorg.com/news/2011-08-dark-illusion-quantum-vacuum.html http://en.wikipedia.org/wiki/Pioneer_anomaly#Possible_causes http://www.technologyreview.com/blog/arxiv/23198/ http://www.technologyreview.com/blog/arxiv/27102/?ref=rss http://www.sciencedaily.com/releases/2010/06/100614121606.htm

Some interesting points from these are that one of the virtual particles created from the vacuum might have negative mass, and that the magnitude of the Pioneer anomaly seems to be proportional to the Hubble constant times the speed of light, and that cooper pairs in superconductors might not be affected by gravity because they are in a delocalized quantum state, and that antineutrinos oscillate at a 40% different rate than neutrinos.

I’m wondering if since neutrinos oscillate between different flavors, maybe they spend part of their time in a kind of indeterminate state unaffected by gravity and so can take a shorter path through the earth than light can, because light has to follow a curved geodesic. The straight line distance over the curved distance probably closely matches the discrepancy found by CERN.

This would make sense, because if you think of any particle or photon as having a certain amount of mass-energy equivalence that determines the path it takes near a gravity well, then if a neutrino is oscillating between different masses, it has to borrow that energy from somewhere, probably from the “vacuum” somehow. It may briefly create a negative energy virtual particle near it when it transitions from a low mass neutrino to a high mass neutrino. The negative energy virtual particle may be able to move faster than light because it would have imaginary mass. When they recombine, one would imagine that it would happen at their center of masses, which would be farther ahead in space than expected. A few oscillations over 732 km could push the neutrino ahead by several feet so it arrives early.

Another thing that might be happening is that if the neutrino is exceeding the speed of light, then it will also show an antigravity effect by following a straighter path than light, so arrive slightly higher at the detector than expected. Measuring this difference in angle might be more accurate than finding timing discrepancies. It would be as if the earth was pushing the neutrino out slightly along its path, so that new path would be travelled in the same amount of time as light would travel along its fastest possible geodesic.

Some quick tests of these possibilities might be to pass neutrinos all of the way through the earth, so that they are repelled the same amount in both directions as they pass through the core, or to pass them through longer portions of earth and see if the discrepancy is proportional to distance, or to build a detector in space and see if a straighter geodesic (due to less gravity nearby) reduces the discrepancy, or to see if a path tangent to or perpendicular to the direction of gravity affects it. Any of these results, whether positive or negative, would be revealing.

Now I fully admit that I only have an engineering background and not a physics background, but there are some fundamental concepts like mass and energy that should still work no matter what is being discussed.

Why is all of this important?

I feel like these articles are all hinting at something profound that can’t currently be explained, sort of like how Einstein’s quanta of light description of the photoelectric effect led to quantum physics.

One ramification of this is that if neutrinos have mass but don’t interact with gravity or matter because they are in a kind of indeterminate state, that might lead to a difference between gravity and inertia. Because our current understanding is that if you have a black box of some mass or energy that has the same mass-energy equivalence as something else (for example a charged battery or a dead battery plus a small additional weight to make up for the missing energy), then they will follow the same path through space near a gravity well.

But if neutrinos or cooper pairs or teleporting electrons show that mass can be converted to an indeterminate state that doesn’t interact with space, time or gravity in a normal way, then we would have a way to control gravity. That’s because we could put some kilograms of superconductor in one box and some kilograms of normal matter in another, and if we just lower their temperatures to the superconducting temperature, they would follow a different path through space, differing by the mass given up by electrons that have become cooper pairs. This will be a small effect, roughly in proportion to the number of electrons in an electric current vs avogadros number, so maybe a coulomb over a mol, or perhaps one part in 10^5. But it should still be measurable. We should see similar effects with superfluids and maybe even Bose-Einstein condensates.

From what I understand, they haven’t found a mass difference yet in superconductors, but perhaps that would change if they were moving at high speed. Perhaps frame dragging doesn’t occur for the cooper pairs. Maybe inertia would differ slightly from gravity. I don’t think those sorts of measurements have been done yet.

And for me, this fringe stuff has always excited me the most, because it might hint at new kinds of physics beyond relativity and quantum mechanics. I don’t really feel that string theory has panned out, or any purely-theoretical construct for that matter. We need some hard evidence to start from, and these sorts of experiments could provide it.

The Nonstarter Problem

Thu, 22 Sep 2011 16:14:03 -0400

My life thus far can be most succinctly summarized as lost opportunity. Like many people, I feel like I have this grand potential just yearning to be expressed, but there are systems in place in our society that make it difficult to work outside of established channels. What do I mean by that? Well to avoid a lot of issues with interpretation, I can state it mathematically. Our lives are not subdividable.

We can’t be half a parent, or one quarter a doctor and three quarters a lawyer. There are certain things that are all or nothing, with powerful forces at work that select for people who excel at any one thing. We are a Michael Jordan society and glorify the idea of virtuosos who work at something for a decade or two and finally make it big, to the esteem of their colleagues. At least in America, this notion of striving for something is practically written into our cultural DNA.

But I strive to be a renaissance man. I don’t really care about fame or power, or even money for that matter. I’m interested in hundreds of things and could spend a lifetime dabbling in a bit of this and a bit of that. One of my favorite quotes is “specialization is for insects.”

The irony is that I have the training to be a successful engineer. But I’ve been spinning my wheels for a decade now trying to be an entrepreneur. I’ve never really had the luxury of working at a job that I felt was sustainable, making an impact and pushing my potential. I’m envious of people who have that. I feel like a fraud.

So let me get to the point. I have not had a traditional career, because I feel that being a good engineer would only represent perhaps 10% of what I want to do with my life. Anybody feel the same way? I imagine we’ve all had that soul-sucking realization, deep into a workday, where we would simply like to be doing anything else than what we are doing. But we can’t, because of debt, or children, or other seemingly endless obligations.

Or because the money is good. We’ve worked hard our whole lives for this job, and what are the chances of finding something else this secure where we can make similar money? So we hold onto the job, for years even, and allow its pressing urgency to consume the other 90% of the hopes and dreams we may have once had for ourselves.

I’m finding that my worst career mistakes have been painful in proportion to my bending to someone else’s will. It’s like that 10% fulfillment that I was holding onto got reduced to 5% or even fell to zero when I was doing the most menial work (in my case moving furniture in the early 2000s).

Our jobs pay well, but at what cost to our psyches? If only there was a way to transition from an all-or-nothing society to one in which we tap human potential to create an order of magnitude more health and happiness than we have today…

That may just be starting to happen. After a lot of years of watching this stuff, I’ve noticed a shift in the freelancing field. These are awesome:

Hirelite

Coffee & Power

Tinyproj

But this is AMESOMER:

GroupTalent

When I saw that site yesterday, it’s like fireworks went off in my head. There’s been a transition from the idea of an entrepreneur as a lone wolf to one in which he or she can contribute slices of time at the going rate for a chosen field.

Imagine a place where freelancers can work together on projects and share the risk and reward. I don’t know why that never occurred to me. This is the concept I’ve been waiting for my entire life. Something between Amazon’s Mechanical Turk and Elance, where an entrepreneur can jump in, perform a few hours of work, and get paid at the going rate for the field. Computer engineers make $100,000 per year, so why can’t I work for four months and earn the $33,000 that makes for a good living in my home state of Idaho? Then I could spend the other eight months working on the pie-in-the-sky ideas that could revolutionize our lives.

Now we can begin to extrapolate. They say real unemployment in the US is somewhere between 10% and 20% if you count the underemployed and people who have stopped looking for work. I think it’s actually higher than that. If we look at current production in America, divided by human potential, I’d be surprised if employment is even 50% of what’s possible. Heck I’d say it’s 10% at most. How many hours a day do you waste at YOUR job? I’ve often felt that on any given day, I only get one hour of real work done. Two at most. That’s just 25% productivity. And that doesn’t even count the support staff of administrators that increase GDP but are non-producing as far as providing goods or services.

For businesses like call centers and Walmart that profit from “unskilled” labor, they are tapping just 1% of their human potential. Categorizing someone as unskilled is like calling someone a moron. It’s unjust because it removes the element of possibility. I’m a firm believer that every human being has not just a skill, but a multitude of talents that go far beyond what others see. If they had to fulfill that potential to save the world, maybe I’m naive, but I truly believe that most people could do it.

And they might just have to, the way things are going. The current state of things is self-evidently not the future that my generation dreamed about while we were growing up. The sociopaths are too powerful and are over-expressed in the business world. Public and private debt are so high now that they probably can’t be paid off, and the fact that politicians aren’t up to solving the debt issue doesn’t help. We are in real trouble. And Obama’s jobs program, as optimistic as it is, doesn’t address the problem of people opting out of “drudgerous” work.

I’ve always dreamt of having a small lab with a dozen other geeks, where we design and build disruptive technology. Instead, I’m currently an out of work deadbeat trying to scrape up enough money to avoid bankruptcy. On paper I’m a complete and utter failure. But how many great inventors and artists were perceived as failures in their day? That thought is what drives me to try to see a little further and imagine greater.

So maybe if we focus on the nonstarter problem - people like you and me who would love to make something of themselves but so far have not had a viable path towards their goals - we might finally see some real progress. Because the problem isn’t working harder or taking on more responsibility. That stuff comes naturally with a love of something. It’s cultivating imaginations and sparking innovative movements that lead to prosperity and seemingly impossible things like the moon landing and high temperature superconductors and someday fusion, youthful longevity, artificial intelligence, a cure for cancer and all of the other things that are not likely through existing means. And the best part is, businesses will have a larger, more engaged, more dedicated labor pool from which to draw from.