I’ve seen a lot of myths about There.com floating around, and I thought I’d take some time to dispel some of them.
Myth #1: There.com closed because the “Physics” in the world made it too expensive to run
Truth Rating: Not Entirely
One big difference between There and other virtual worlds was it’s “distributed physics“, which basically made sure that everyone saw the same thing happening at the same time. This isn’t to be confused with “Physics”, which many game engines have, but few (if any) have true distributed physics (There’s distributed physics are patented, by the way).
To do this, There has to simulate the activities of all the objects in your “sector” (the area of the world you were in) (including all other avatars) and distribute the results to everyone around you.
This meant that, for example, the Cosmo Girl server(s) had to simulate the 50 or so avatars that hung around there, along with any other objects which happened to be in the sector like benches, buildings, fire pits, trees, you name it. We have to simulate the physics of the stationary objects like trees because you could collide with them.
It is kind of dumb that that tree is an object, and not part of the terrain, but the current terrain system didn’t support that. It was a mixed blessing. More on that later.
(Trees weren’t so hard, since they knew to be still and not slide down hills. Things like buggies, feelium balls, etc – they were always being simulated when they were “out and about”. Buggies used to have this problem too — actually 5X the problem since it simulated the 4 wheels and the buggy body independently (actually they were called “Diphy Balls”) — but SamSyn fixed that by making “sleeping buggies” which didn’t do so much when they were standing still).
Many people have claimed that the reason There closed was that we had to run too many servers because of this.
This wasn’t true. If the world had been tremendously (uncomfortably, even) crowded, it might have been true, but the fact was, many sectors were huge (since they had so few objects and avatars to simulate), and most sectors had very low occupancy, most of the time.
Also, by 2009, or even 2005, hardware costs had fallen so far that the simulation costs ceased to matter. In another post I’ll make, I’ll explain how, if we re-bought all the hardware for There today, it would cost a lot less, take a lot less space, and consume a lot less power.
I might have been responsible for this myth. In the last week There was open, I spent time in world talking to people, and to illustrate how expensive it is to run There, I talked about the monthly ISP bills, which is what it took to run those servers. That was a big number, but we also had to pay salaries, rent, etc, etc. Just getting rid of the physics wouldn’t have kept There open.
So, no, the cost of physics is not why There closed. See below – all told, servers and bandwidth were less than half our expenses every month.
Myth #2: There.com closed because it used too much bandwidth
Truth Rating: False
One of the design points for There was that it had to be able to run on 56Kb dial-up lines. Remember, when There was designed in 2001, there was far, far, far less broadband than there is today. If anything, it’s better than it was then because since 2005 we made several improvements to how we used bandwidth.
Where did this story come from? When you go to a new area in the world, or see new content, There would download the content (a.k.a. “assets”) so you could see it. This uses bandwidth (even though it tries to not interfere with your experience), and it could cause a problem on lower speed lines. There has to be able to download fresh content if you want to have dynamic content in the world.
You don’t see this in video games because all the content comes on the CD (though WOW is introducing this feature in the upcoming Cataclysm release).
(I find it very amusing that people called There “obsolete” yet World of Warcraft just introduced the idea of streaming content into the world, 9 years later. Not to say that WoW isn’t a fine game.)
The traffic between your client and the servers (your client was talking to it’s Avatar Manager, the server for the sector you’re in, and perhaps the servers for one or more surrounding sectors at any one time) may be what people are talking about when they talk about “too much bandwidth”. This still bothers me, because all that was was physics and visibility updates which were sent in a highly compressed form (not long sentences like “Your Avatar moved 4 feet on a heading of 237′ 15″ and you can now see that tree”)
Practically though, bandwidth was a tiny fraction of our expenses. So, that’s false.
The moral of Myths #1 and #2 “No, it wasn’t that the There software was too expensive to run”. All of There’s infrastructure costs we less than 50% of our costs of operation.
Myth #3: There software was a mess
Truth Rating: Sort Of
This myth is probably dependent on your perspective in the company. Here are some things which are true:
. By necessity, There’s software is very complex. It is a 3D Game engine, a distributed physics engine, a distributed content provider (so everyone could see new, changed, or removed content, or objects in the world), an event manager, a commerce engine, an IM system, etc, etc, etc. It’s was composed of millions of lines of code, and was very complex.
Did that make it a mess?
It made it complex, that’s for sure, and we had lots of pieces talking to each other, and we didn’t spend a lot of time on documentation (just like any software company). All this complexity made it hard to make some changes to There, which might lead anyone to decide it was a mess in frustration.
. There’s software did a lot things which had not been done, or done well, until 2003, or even now.
Take an example: Crossing Servers. Imagine you’re on a hoverboard, going full speed across a sector (which is on one server). At some point, your avatar, and your vehicle, and anyone riding with you, will have to cross over to another server. So, not only can you see across the line to the other server, you’re going to cross it in a fairly smooth fashion, and alll your information is going to migrate from one server to another. Trust me, that hadn’t been done before, and I’m not sure it’s being done now. Did that all work perfectly the first time? No. Did we sometimes think it was a mess and wish we hadn’t done it that way? Yes, but when you’re frustrated, that’s what happens. At the end of the day, if it wasn’t hard to do, There would have not been There.
. Some things written from scratch which could have been gotten off the shelf. More things than I was comfortable with, that’s for sure. It could have been that, when those parts were written, the off-the-shelf solution wasn’t quite mature, or well-known, or even documented properly. In 2001, or even 2003, the Open Source community wasn’t as robust as it is today, so it’s not surprising we missed some opportunities there.
. When we first launched There, and right up to Black Friday, we were still learning how to run it, and those same brillant engineers were still learning how to write software which could be run in a production environment. Thinking back, Second Life and There were about equal in that regard, so everyone was learning about these things then.
Many of the folks who took over There in 2005 were from production environments, so one of the first things we did was focus on stability. It was only a few short months from going from scores of server crashes a day, to none. We continued to improve the product’s stability over time, but don’t take my word for it: When the economy was good, it was good enough for MTV, Coca-Cola, and a few others. Virtual MTV, which ran on the There platform, appeared regularly on the big screens in Time Square live. Not bad for “a mess”.
I can see on of the ways the “mess” myth got started. One of the things which There was, and still is, bad at is the Art Path. This is how artists get content from their tools (3DsMax, Maya, etc) into There. This applied to all art, from the Terrain engine to models to layouts to houses to avatars to…well, you name it.
Actually, no, “bad” isn’t a strong enough word. Let’s say “horrible”. Or even, “getting art into There was like trying to remove your own appendix without anesthetic using nothing but dull pencils”.
Everyone who has worked with There knows it’s a nightmare.
Why is it like that? Because, at the time, the management (which wasn’t me, I was just busy investing at the time), focused engineering resource on the user experience and product, and unfortunately sacrificing artists in the process. So, if you were an artist, you might get the impression the Art Path was just a mess for No Apparent Reason.
Trust me, not a universally loved direction at the time.
(To be fair, again in 2001, or 2003, things like standards for 3D graphics formats (such as Collada didn’t exist, so we had to “roll our own”.)
I have worked with There’s code base since 2001 (when I started investing and working at There). Is it perfect? No. Would I change it? Of course. Are there things in it which are just plain wrong? Of course (and I probably put most of them there).
In my experience, software that is a “mess” is software you can’t improve, or fix. I’ve known of such software, and suffered through trying to fix it, before just throwing it out and starting again. Given that we spent 5 years at Makena improving There, it’s doesn’t qualify as a “mess” or an “unfixable mess”.
Myth #4: There software is outdated and obsolete
Truth Rating: False-ish
What does “obsolete” mean?
Long ago, I worked at a then-tiny company called Oracle, which made databases. We released a new database about once every 2-3 years (I was there for Version 5 and Version 6). Do you think Oracle replaced every line of code with each release? Of course not! In fact, I’ll bet you if I pulled Oracle off the shelf today, I’d find code from Version 5 still in it (probably the dreaded UPI interface, come to think of it), and maybe some from earlier. And guess what? Oracle is still doing SQL-based Relational Databases, 15 years later.
Is Oracle obsolete?
Lots of people look at There’s graphics and say “Wow, There is outdated. The graphics on my XBox or even in Half-Life are much better”.
Well, of course they are. There’s target platform was any PC you could buy in 2003, with even with integrated graphics. To make a product which worked on those machines, and at 56KB dial-up, we had strict, strict, strict limits on the complexity of the art (vertex counts), and texture memory.
But that was the art. Let’s look at the other pieces:
. Rendering engine. There’s rendering engine supports Direct X 8, Direct X 9, and OpenGL on Windows and OSX. Other engines, like the Crytek Game Engine, support…Direct X. When we did DX9 support, we also re-did the engine’s innards, so everything is now a “material” and we support things like specular reflections and modern shaders. Sure, if we were on DX10 we’d be “even more modern”, but what other “virtual world” supports DX10 today? I guess we could try and make There run in Flash, but that doesn’t seem like a step forward.
. Terrain Engine. Ok. When There was built, we used a fractal based terrain engine, which was all the rage at the time. It was good for “automatically generating” lots of terrain, while giving you some control over how it looked. I say “some” because, since it was fractal based, pulling on one bit could have…unexpected affects on other places, like trying to tighten a cover on a bed — it straightens out in one place, but wrinkles up in the other. It was computationally not ideal either, for both the client, and server.
In 2008-2009 we had the engine re-built, and they built a brillant model-based engine which is now used in Olive. We never got the chance to convert all the terrain in There, but that engine is available for use.
So Terrain engine, obsolete? Well, not-what-we-would-choose-today is what I’d say.
. Object system. Everything in There was an object. Why? Well, remember, there has physics, and objects needed to interact with each other and avatars, and that was done by making them objects. Also, remember that every object in There could be replaced on the fly. If we’d melded every thing in There into one object, then we couldn’t very have done that, could we (it would be kind of like melting two pieces of ice together into one piece, and then trying to precisely unmelt them later to take them apart). I’m sure that today there might be algorithms we could use to do this, or even declare some objects “not replaceable” but that wasn’t an option at the time.
Objects? Obsolete? Well there are lots of way to make them better — in fact in 2005/2006 we introduced new lighter weight objects so we could build denser environments for MTV — but I’m not sure that’s “obsolete”. If you think so, go build a system which can replace items on the fly like There can, and report back on how you implemented it. I’ll bet you used objects.
I could go on forever about each piece of There which people have called “obsolete”, but my answer is always the same: Yes, there are pieces which need to be upgraded, sometimes to keep up with technology, sometimes to keep up with how they are used (like inventory, for example), sometimes because they never deserved to live. Does that make them “obsolete”. Hmm. Well, I guess so. The alternative to “obsolete” is you re-write or upgrade the entire engine to make it “more modern” (whatever that means). It took several hundred man years and millions of dollars to build There the first time. I’m not sure that you’d be justified in doing that again so you could say it was all now “modern”.
Myth #5: There’s Developer Sale systems were terrible
Truth Rating: True!
Disclaimer: I wrote all of There’s Auction software, and the Developer selling systems (and a lot of other bits).
Let’s look at how the Auctions, and it’s evil twin, the Developer Program, came into being: Sometime late in There’s pre-Black Friday Life, we decided we needed a developer program, and a way for people to sell those developer items. And we needed it now. There were enough folks to work on StyleMaker and Previewer, but no one to work on the selling end.
So, they came to me, and said “Could you somehow make auctions do this in one month?”. I thought about it, and said “Sure, I can do that”. Of course, I’d forgotten I was about to leave on a 3-week vacation in Asia when I made that promise, so most of the Developer Sales system was written in Asia, not Palo Alto. There are actually pictures of me sitting in a hut in Phuket, Thailand, working on the auction code.
When you understand that it was based on the auction system out of necessity some of the stupid things begin to make sense:
. Auctions are design to run for a fixed interval of time and there are lots of things going on under the hood to make sure that works well (I knew a little bit about building auction systems). Unfortunately some of these things were the opposite of what you’d want in a Store-like system, or a system where you only listed something once.
. Auctions were built around re-selling things in the There store. Thus it was tied very strongly to the There catalog hierarchy. This worked ok for a while, until the developers starting using the developer system to make things which weren’t anything like what was in the There store. But, since it was all tied up in this long chain between “Ok, you used a thing which looked like a buggy model in the beginning, so we’re going to make you put it in the category with buggies, even though it looks like a flying horse”. That’s why the categories were so messed up.
. Remember my story about “Art Tools”. Well, Commerce got it too. Instead of getting proper tools to manage the categories, etc, we ended up with this huge spreadsheet, and this byzantine, error prone process called “SCHMO” to manage it. PLUS the whole tie-in to the Previewer, etc, made it even harder to maintain developer categories.
(I didn’t work on the last two bits, just the auctions)
So, at the end of the day, we had to build the whole thing in a month, and then we didn’t have time to really do any major upgrades so we moved on to other things.
In 2009, we actually did a redesign of the There WebSite, including Auctions, and I actually implemented a lot of the code for the auctions changes. There’s a branch of the code base out there that solves a bunch of these problems, including things like bundles of products, picking categories properly, using tags (keywords), etc. Sadly, we never got the time to do all the CSS and HTML work for the front end side. Code with no clothes.
So yes, I agree that bit was all broken, and it was my fault.
(In a small defense, the Auctions and Commerce system were, mostly because of the Developers, one of the major ways There could make money).
Myth #6: There was forced to close by Law Enforcement
Truth Rating: False
This is completely false. We enjoyed an excellent working relationship with Law Enforcement, including the FBI, and always cooperated with them when asked, not even waiting for the subpoena to produce information. We were mentioned favorably (well, as favorably as any other virtual world) in the FTC’s Report on Virtual Worlds, in fact they were shocked that we were so eager to work with them.
At the end of the day though, the whole myth is just, well, stupid. Do you think the attorney general would spend thousands, or hundreds of thousands of dollars to investigate There, and then “let us off” if we closed the service? Not likely.
So, no, There was not forced to close by Law Enforcement.
Myth #7: There made millions of dollars in profits
Truth Rating: False
I’ve said this before in comments, but I’ll say it again: There.com never broke even.
From the day it was founded until the day it closed, it operated at a loss every day. Thanks to it’s investors who believed in the service, and employees who were willing work at start-up wages, and members who had faith in us, it was able to stay open, but it never made a profit, much less millions of dollars.
I’ve talked about this in replies to comments to my other posts ad-nauseum, so I won’t cover that ground again here.
Myth #7: There can be run on one server
Truth Rating: False
There can be run on one server. If you’re an engineer working on There, and want to have one, maybe two avatars in the world.
Otherwise, here’s what you need:
- An “Avatar Manager” for every 5000-10,000 members.
- A database server with RAID storage (so you don’t lose all your content if a server goes down
- A physics server for each sector in the world
- A visibility server for each sector world
- Web servers for web apps and static content
- A load balancer to sit in front of the web servers.
- Spares to swamp in when one of the above fails
- Backup servers to catch and store the backups each night, especially for the database servers
- Secure servers for billing.
- Firewalls (real firewalls, mind you)
- Etc, Etc
You could combine or eliminate some of these, but you would regret it eventually. For example, if There ran a pay service, and was down for 24 hours because there were no spare servers handy and one failed, I guarantee, people wouldn’t “just understand”. The same would happen if you “eliminated” the firewall, or had insecure billing servers, etc, etc.
Some people who illegally stole There Beta software have claimed that they could run it on their computer with no lag, and therefore we should have been able to do that with the service. Since the stolen software did not have a physics server, a visibility server, a proper avatar manager, or any of may other pieces in it, that would have been kind of difficult.
If There could be run on one server, or even just one rack, it would be open today.
Myth #8: Many offers were made for There, but Michael Wilson turned them all down.
Truth Rating: False
I did not get a single legitimate offer for There.
I did get an offer from someone who wanted our member list for an adult-oriented virtual world for free, but that was it.
I’m not sure why anyone would think that, after investing millions of dollars in There, I wouldn’t jump at the opportunity to recoup at least some it through a legitimate sale. If someone believes we turned down offers for There, they really need to prove it or stop saying it, because it’s a lie.
(In the interests of complete disclosure, there is at least one group who is so morally bankrupt, we’d be hard pressed to accept their offer. I’m not going to say who they are, but I think everyone here knows who I’m talking about).
Myth #9: The fact that There did refunds and Therebucks BuyBacks when it closed is proof it was profitable.
Truth Rating: False
As above, the investors wrote checks every month to keep There open. The refunds and buybacks came from the same place.
Myth #10: There should have refunded every cent people spent on Therebucks when it closed.
Truth Rating: False
I understand why people feel this way.
Let’s look at where that money went. All the money that people put into There then went to paying things like ISP fees, utilities, rent, paychecks, insurance, benefits, hardware costs, software licensing costs, attorney’s fees, you-name-it.
In order to “pay back” all that money we would have had to go back to the ISP, the Electric company, employees, insurance companies, hardware vendors, software companies, and attorneys, and say “Sir, can we please have that money back? We need to give it to back to the members.”.
We would have loved to be able to do that, but I think everyone agrees that wouldn’t have worked. There’s no business in the world that I know of who can retroactively refund all the money you’ve spent if it closes. If they claim they can do that, then they must have some pretty unique relationships with their ISP, electric companies, … you get the idea.
I’m not trying to be flip here, but that’s the reality of the situation.
Myth #11: There could have stayed open if it had just…
Truth Rating: False
There are lots of ways people think There could have stayed open. Popular ideas were:
- Convert to a Pay Site.
- Shrink the world to one island.
- Drop the P.G.-13 rating.
- Allow copyrighted content
- Pay people to refer their friends.
Sadly, none of these would have worked. We know because we tried many of them. Some of the other, well, it’s not in our DNA to make an adult site, or a site that ignores people’s intellectual property rights.
In 2005, when Makena took over There, it was a pay site, and it was much smaller. And it still didn’t break even.
We attempted a “for pay” Refer A Friend program in 2005 or 2006 and got I think one referral. We tried it again in 2010, with the same results. Personally, I would have loved for that kind of program to work (which is why we invested in the backend support for it).
The fact is, to run a PG-13 world, and provide you with things like Community, Operations, Customer Support, Developer Support, ongoing engineering, QA, etc is not free, or even cheap.
We could open a world without one of those things, but not for pay, since customers would rightfully take umbrage at the fact we didn’t have quality “…name your service here…”.
There could be a way for There to work, but it would take investment in new hardware, and significant changes to the platform. I’ll outline those in another post.
Myth #12: There could have stayed open if it had just gotten more sponsors
Truth Rating: False
We actually had full-time salespeople and business development people who did nothing but pound the pavement looking for sponsors. Here’s the cold, hard, facts: Sponsors are looking for advertising-levels of impressions, or eyeballs. That’s not thousands of people, or even tens of thousands, it’s millions of impressions. I’m sad to say that no, or very, very few, virtual worlds offer millions of impressions to advertisers, that’s why so few survive.
If that wasn’t a big enough obstacle, the recession absolutely clobbered ad revenues, everywhere. You can imagine, that if your revenue is down 25% or even 50%, are you going to spend on a known, traditional ad platform, or a new, unproven one?
Again, I wish we had more sponsors, especially cool ones like museums and zoos and you-name-it, but they just weren’t there to be had.
Myth #13: There closed so it could making millions working for the Government
Truth Rating: False
Anyone who thinks this is clearly not familiar with how the economy has impacted Government Funding, especially for education. Ask a teacher if you don’t believe me.
How bad? I’ve been waiting for six months for a new license plate from the State Of California due to budget cuts.
Myth #14: There didn’t care about it’s Members, that’s why it closed
Truth Rating: False
This is one of my favorites, along with “There didn’t even try to stay open”. There tried to stay open for seven years.
I think all we can do in response this is point to the evidence here, and everything else we did, and after that, it’s up to you to judge.
Myth #15: 3rd Party Currency Resellers had no impact on the There economy
Truth Rating: False
In an analysis we did in 2009, we found that 25% of the currency purchases in world came from Third Party Resellers. That means the revenue didn’t go into There’s pockets, it went into someone else’s.
All I will say about this is that we should have taken a stand on this much earlier than we did, and I take responsibility for not doing so.
Third party re-sellers can play a part in a Virtual World’s ecosystem, the question is whether you let them play it, or you do it yourself. I just provided you free research data on what happens if you let them do it for you :-).
If you decide to open your own virtual world, please take note. I’ll be happy to talk to you offline about how you’d eliminate this impact on your economy while still giving people a way to get real money for virtual goods.
Myth #16: There will never reopen
Truth Rating: Who Knows?
If you know this, then you can either see into the future, or it’s wishful thinking on your part.
I don’t know that There will never re-open. I spend time every day thinking about what it would take to re-open it, which I’ll talk about some day. But it’s not a simple matter of just making it smaller, or making it a pay site, it’s a lot of things.
There has been in the Virtual World business for ten years, some of us for more than that. If we need to think hard about how to make it economically viable, then it must not be a very simple problem (either that or we’re really, really dumb).
As I said, I don’t think anyone knows if There will ever re-open. I keep it open in my mind and my heart, and maybe we can find a way to open for real again some day.
P.S. Here’s a list of other myths which I didn’t have time to expand on:
- “There was based on Internet Explorer” (IE). False. IE was, and may still be, the only browser which gave us COM-control access to “drive the browser” from a program at the time. IE was also the only browser which supported XSL at the time, which we used for the site. Finally, in 2003, Netscape/Mozilla was kind of a mess, and needed for the emergence of Firefox to become a legitimate rival again.
- “There’s text chat was just a patch till we developed voice”. This is one of the most ignorant things I’ve ever heard. We went through something like 32 iterations of There’s ACC (avatar-to-avatar communications, which is “Text Chat”) to get it right. As it was, Voice was developed using off-the-shelf software in about a month, and only because the then-CEO slipped and told a reporter we’d open with voice chat. As it is, we’re really glad we did voice, but ACC is one of There’s crown jewels of technology and creativity.
- “There closed because of Forterra….(fill in the blanks here).” If you read our closing notice, this isn’t true at all. There acquired the complete and unfettered rights to it’s platform in 2009. Forterra had nothing to do with There’s closing.
P.S.S. Wow. 5016 words. Sorry it took so long!