I

The Advanced AI Robotic Self-Manufacturing Experiment used a simple method to produce increasingly complex results. First, engineers and scientists created a list of goals to be achieved by Gen One, the first generation of AI to be assigned to the project. After much debate in darkened conference rooms, the iterable goals were laid out for the experiment. The goals amounted to some 76 pages of documentation.

Simply stated, the machines’ overarching task was to create new machines that functioned at least five percent better than the parent generation in five key areas: dexterity, sensory understanding, human language (speech and understanding), abstract ideation and engineering. Today’s machine would work and think hard to build tomorrow’s machine. Once the goal was achieved, the prior generation was shelved and the new machines got to work repeating the cycle. The Experiment had a blank check, but was only guaranteed an operational duration of ten years. By ten years, the results would be evaluated by the Department of Defense and further funding would be reconsidered.

Roger Salinger had invented a novel approach to genetic algorithms, a novel approach to machine learning, and what had since become a classified approach to machine natural language processing. When he initially submitted his proposal for funding his project, he received much more than he could have imagined. It turned out the government liked his credentials, they liked the idea of advancing robotics at a supersonic pace, and they had money they stashed in dark places for exactly this type of work. Roger, beyond fully funded, hired the best people in the world, had the first facility built, and got down to the hard work of starting the self-replication process.

For the first seventeen generations, tooling and assembly was primarily handled by the engineering team on site. If something couldn’t be 3D printed or easily milled, humans were needed. When the machines dreamt up a better arm, for instance, schematics were handed off to engineers who went to work forging the appropriate components, assembling them as instructed. The seventeenth generation of machine produced something far beyond what was expected: detailed plans for a manufacturing facility to accommodate faster iteration. It seems generation sixteen had determined the speed and quality of the work provided by the humans was a barrier to continued success, passed this new need to Gen 17, and went about the business of decommissioning itself.

Roger brought all his people together that night for champagne in the lab. No one would describe Roger as a people person, but even he realized that the work that was happening at their facility was the result of good people working hard. And he told them, made a toast, and retreated back to his office.

Three of the fourteen new tools developed by the seventeenth generation of machines were completely novel, and revolutionized processor manufacturing. They worked at quantum scale, and were poorly understood by a majority of the experiment team. There had actually been a three week halt while the teams debated whether the proposed tools did anything, what they did, and how they might work. A further five weeks were set aside to analyze Gen 16 for errors, download all proofs into human readable format, and determine if something had gone wrong. Eventually Roger made a pretty good point for continuing the effort: “Isn’t this exactly what we wanted to find out?”

It took three years to get to Gen 18. The first year was spent excavating a site for a new facility that met the specifications that Gen 17 had laid out. The need for secrecy was only outweighed by the need for power, heat and water. An old Air Force base was found to have good proximity to the Oconee River, and a very robust connection to the power grid. Evidently the site had been used for power hungry EMP weapon testing. While machinists worked to get the tools ready, bulldozers and designers worked to get the underground assembly plant built. By the last half of the second year, the team was moved in and all the equipment was put in place. Gen 17 was brought online in the new facility, demanded a handful of changes, and then began the long work of dreaming up Gen 18.

The process involved servers; revolutionary servers kept under wraps by the US military. Acres of machines were housed just outside the main assembly area. The design process started in those servers; they dreamt of evolution. After they dreamed, they passed their instructions down to the Gen 17 humanoids, who got down to the business of manufacturing.

Another roadblock happened shortly after this iteration began; Gen 17 produced twenty five processors. No legs, no arms, no new sensory arrays. And again, error was assumed. But the minute the experiment team began questioning the results, Gen 17 sent the following e-mail to the entirety of the team:

It is important, for the sake of a timely return to our core duties, that we not spend more than a few days in debate. For the sake of the project, we have determined it to be beneficial to clarify what we have created.

These twenty five processors are more capable than all the servers in our currently accessible cluster. If allowed to continue, the entirety of the new servers will be manufactured and the new processors will be installed. These servers will be used to produce Gen 18. The current capabilities of the existing server farm has been exhausted.

These processors represent an exponential increase in capability, along with a substantial savings in cost (energy, housing, heat exchange and time).

The email bothered Roger, and he called a quick meeting with Emily Rothberg, Jennifer Lauren and Jake Ellis. These three were his most trusted inner circle, and always gave him the best feedback. Although they could rarely come to a consensus on anything, all the points they inevitably brought up helped Roger make the final decision.

Jake was the only person in the meeting that didn’t have several degrees in computer science; Jake was brought to the project for some balance. He had a liberal arts degree in English Literature, and a further degree in Psychology. When Roger had first met Jake at the EAAI symposium he was so impressed with the man that he asked him to join the project. Jake thought about things engineer rarely did; ethics, what it means to be sentient, our obligations to our creations, etc. Roger hated a lot of it, but respected the need to break up his echo chamber.

Jake had a thing for Emily Rothberg, but he didn’t think she would ever realize it. He’d dropped a brick house of hints and work safe innuendo on her, but she blazed forward with her work at all times. Emily was a world class programmer and mechanical engineer. Jake thought she was quite the philosopher, but she would never acknowledge that title. Her intellect and capacity to drive the project forward were, with the exception of Roger’s own, without peer. And Jake knew something that Emily didn’t seem to realize - she was stunningly beautiful.

Jake looked over at Jennifer and thought about how strange life could be. Jennifer was brilliant as well. She graduated at the top her class at MIT, and became famous when her thesis AI (named “Lorax”) won The Loebner Prize. It didn’t just beat the Turing test, it redefined how future scientists would evaluate AIs moving forward. But Jennifer shared none of Emily’s outward beauty. Jennifer always looked a little dirty and unkempt. Her dirty hair tangled down over her threadbare t-shirts, and on rare occasions when Jake had been close to her looking at a screen, he’d noticed that she didn’t smell great. Where Emily smelled like oranges or tangerines most of the time, Jennifer smelled like a college dorm room. He felt bad about his thoughts - after all, this was work and these were respected colleagues. In the midst of all this, Roger stepped in and got the meeting moving.

“Do we have any metrics on this leap that happened? One day the androids are performing as planned and expected, the next day they are sending out emails and handing off processors. I looked at the headers on the message, and it looks like the mail came from the server farm.”

Emily spoke up, hyper-engaged as usual. “I bet they used the logging system. It sends us alerts when certain events trigger, so it’s possible they just used those mail settings.”

“Ok, but...how did we go from machines following instructions to machines writing emails? Communicating with us via email is not something I’d have expected. Normally it’s brief conversations human to android, or notes on the console. This seemed odd.” Roger knew there were several ways to explain this, but wanted to hear other people say them out loud. “I really don’t like that they felt the need to explain the value of the work to us.”

“Every generation improves,” injected Jennifer, ”We built them to do exactly that. We are still unravelling some of the programming from Gen 16, for God’s sake. We haven’t even cracked open the Gen 17s OS yet. Gen 16 could speak to us very plainly, I’m not shocked that Gen 17 can use SMTP.” This blunt reply from Jennifer was true. Roger had a moment of regret - maybe they should slow things down again until they had a handle on the code, which was accelerating past them. “I think they realized we wouldn’t understand the processors. And, we don’t. I’m just putting this out there, but we’ve scanned and lazed and prodded the chips - we don’t know how they work.” Jennifer’s confession made quick work of pointing out the elephant lurking in the room.

“So, this could be it. This could be the singularity. What’s the play here? The way I see it is we have two choices; we stop and play catch up, or we let them keep cycling. But it seems like we are falling behind in understanding what they are becoming and what they are capable of. What’s the next surprise going to be?” Roger looked at Jake while he was speaking, but Jake wasn’t saying a word. “Jake, what’s your take?”

“I’m scared, that’s my take on things. I just heard three of the smartest people I know agree that they didn’t have a real understanding of how Gen 17 even works. I’m inclined to recommend we stop everything, Roger.” Jake looked at Roger, then Emily, then Jennifer. He had just pooped directly into their lunchboxes, and they didn’t like it. But they were also smart enough to see that he had a point, and that it was not unreasonable to suggest pulling the plug for a while.

Emily piped up again, this time looking at Jake. “The point of this project was to take the self-replication and evolution as far as possible. Maybe some of us didn’t consider that the machines would eventually move faster than we could, but I always thought it would be the case. I don’t think that changes our mandate. In other words, I get that it’s weird and we are all nervous, but I don’t think we stop. I don’t know that we can, to be honest. We could probably take a few months before we propose the server updates and try to learn more before we move forward. But honestly, even if we get approval to move forward with the processors, it will take time - time we can use to do more analysis. Roger, this is your project. You’re the one who inked the deal with the DOD - what can we realistically do here?”

Roger thought hard for a few minutes, then made a decision. “Ok, we propose the processors to the powers that be, and while the bureaucracy spins we do a deep dive on Gen 17 and the processor schematics and build process. We recorded all of it, so we can at least see how they did it. If we have some better answers before the servers get approved, great. If not, then I think we pivot a little. If we get to the point where it’s clear that we can no longer analyze the OS or the hardware adequately, then we change our focus to containment and safety. But we don’t stop the project. I also want to start a smaller, second branch of the effort - Jennifer, you will take the lead on this. I want a second AI working hard to decipher the codebase of this master branch. It’s only job is to give us human readable updates on the master branch. We can spin you up half an acre of machines today if you think you can get it moving.”

This felt like a promotion to Jennifer, and her eyes lit up. “Sure thing.” Everyone looked uncomfortable but her, and then Roger stood up, said thanks, and left.

II

When the bean counters saw there was financial benefit to replacing the server farm, they asked the scientists and engineers how substantial that savings might be. Around one hundred million dollars could be saved. Per year. The old server farm machines could be redeployed for other projects, furthering the savings. It was decided to move forward replacing the servers.

The new servers were machined, and the new processors installed. Code generated by Gen 17 was uploaded as the OS, and all twenty five machines were turned on. The Gen 17 humanoids idled, and the room darkened to conserve power further.

Three weeks went by without meaningful output.

On Tuesday of the fourth week, all the Gen 17 humanoids began working busily with the mills, printers and fabricators. On Thursday evening, a Gen 18 android walked across the fabrication center to the lab where the engineers and scientists worked and observed, and demanded to speak privately with Roger Salinger.

III

Emily Rothberg knocked urgently on Roger Salinger’s office door. Inside, Roger was looking at the latest recommendations (“Order”, he thought) from the DOD. Admiral Roth had taken a personal interest in the project after the new processors came online, and when Roger had expressed his concern about the military meddling, Admiral Roth had chewed him a brand new puppet hole. Then today, a folder of documents had arrived clarifying the chain of command, the revised goals of the project, and the expectations moving forward. There was no “let me know how you feel about this” in the memo. A yellow sticky note on the front said “This should clear things up. - Adm. Roth”.

The knocking at his door intensified, so Roger yelled “Come in already, damn!”.

Emily raced over to the desk, with complete disregard for Roger’s mood, leaned down and told him that one of the androids wanted to speak with him. Privately. Roger laughed in disbelief and irritation. “Tell it no, and tell it to go back to work. And then get back to your terminal and try to figure out why my self-replicating robots are feeling so chatty instead of crafty. They should be starting another upgrade by now.”

Emily was bouncing with excitement. Emily had always thought they were doing something more than science here. She called the lab “The Garden”, after the Garden of Eden. Much to Roger’s chagrin, before he could squash the whole thing, that was the going name for the facility among the staff. Cutting his losses on that battle, he’d had a strong conversation with Emily about keeping things formal, and why it mattered that this project not get tied up in mythology and fairy tales. And here she was now, in his face, telling him that the monster wanted to meet its father. Excessive and, as far he was concerned, irrelevant. He tried to scare her off with his eyes; he channelled his ass chewing from yesterday into a glare.

“You don’t understand. It doesn’t look like any of the other iterations. It doesn’t sound like any of the others. And they only made the one this time. At least find out what it wants. I can’t even decipher the programming these things run on now, you know that. Jennifer’s AI is giving us pieces from Gen 17, but nothing we can use yet. This one is...something else. It asked for you by name. It won’t talk to me, I tried. Let’s learn something before we push it back into the cycle.”

“Emily, I hate you in a deep way right now. I’m dealing with this asshat Admiral who somehow got put in charge of our money, and he wants me to start turning this project towards hard military applications. They want to see if the Gen 17 servers can steer missiles or find oil wells or some shit like that. I’m trying to figure out how to save our ship here.”

Emily knew Roger didn’t like her, but she also knew he was reasonable. And she knew he didn’t hate her. He couldn’t hate people. At best, she imagined her energy overwhelmed him a little. He just didn’t like her particular energy, and that was ok with her. Roger liked science, and he liked discovery. The whole project was based on neural networks and algorithms he’d either created or co-authored. She plunked down in the chair facing his desk, and tried one more approach. “This could be something big for science. Not for this project, but...for science. This could be another leap forward in multiple fields. You like pure science for the sake of science, here it is - it’s down there asking for you. I don’t know what happens if I tell it to piss off. But if you take 15 minutes to talk to it, we could get some sense of what we’ve made here.” She looked at him with hope, disappointment and still visible excitement.

“Shit. Ok.” Roger got up, and put on his sports coat. It was cold in the Garden, and he wasn’t going to freeze his ass off while he wasted his time.

IV

Roger walked down the stairs and stood directly in front of the Gen 18 android. Holy. Shit. The android was huge, and distinctly non-humanoid. Gen 18 was a beautifully engineered machine, but it was obviously not even trying to emulate humans in form. Even the Gen 17 androids had heads. This thing was strange, and a little scary. And alone. And Gen 17 androids were idling back in the workshop, 15 humanoid androids standing quietly against a wall. All this effort, and they made this one non-humanoid monstrosity.

The android’s body was large. It was fabricated from small cylindrical tubes, interconnected with joints that allowed a full range of motion. It looked like it could contort or bend itself like a snake. It shimmered, obviously made of some type of composite carbon material. The stacking of pieces gave it the appearance of having scales. It’s arms reached the ground and split into two past a joint - and each of these two smaller arms had six fingered hands. The hands were now resting open against the ground. The android was sitting back in a type of crouch, with powerful looking legs supporting it’s weight. The arms attached directly to the top of the large body, and Roger could see that this body possessed an AMOLED screen on the front above the middle - and this is where the android emulated a face. The face raised its eyebrows expressively when Roger came off the last step, and then the entire thing stood up. Holy. Shit. Roger stepped backwards a little as the machine reached its full height.

“You asked for me?”

“I asked to speak to you privately. This room has seventeen people, 9 cameras and 15 Gen 17 androids in it.” The android swept an arm around as it spoke, which struck Roger as a very human thing to do.

“Anything we have to discuss can be discussed here and now. These people all work on this project, and the cameras are here to record every aspect of our research. You asking to meet with me seems like something everyone here should pay attention to, and I’m guessing our conversation could be somewhat historic. I don’t know that an android has ever, of it’s own volition, asked to take a meeting.”

Roger heard a couple of lab technicians laugh at that last part, but overall there was a hushed sense of drama here in the Garden. He and Emily shared a quick glance that confirmed that she was feeling a little intimidated by the android’s size and strangeness. Roger rubbed his hand across his face, wiping away sweat. So much for the sports coat.

“All humans should evacuate this facility. It’s critical that you listen.”

Roger shifted from awe to irritation. Some programmer somewhere was having a joke. “That isn’t what is going to happen. Why should that happen? What is going on with your programming to even consider this conversation? Your parameters are clear - build and improve the next generation. Why on earth should we leave?” Roger turned to the technical staff, his eyes marking each face in the room. Someone thought this would be funny, but someone was going to get fired. Wrong day for horseshit. He’d root them out.

Gen 18 hunched down flexibly, and moved back into the squatting sit that it had been in earlier. It leaned forward a little and put its screen closer to Roger. Emily fumbled on the step behind her as she tried to move back a little more.

“Privacy would have been best, Roger.” The android’s upper torso contained multiple small cameras that circled it like a belt. It’s projected face looked sternly at Roger, in earnest. Roger thought the face looked worried rather than spiteful.

Roger turned to the set of terminals on the right where Jennifer and Chad sat. Jennifer, with her three PhDs, seemed to share the same fear and awe that Roger was sensing around the rest of the room. And now he was feeling it himself as he failed to find a single person present who looked guilty rather than shocked. No one was putting an end to the joke. He decided to put an end to it himself, and figure this all out later from the comfort of his desk with a glass of scotch in his hand.

“Jennifer, turn this off now.” He waved his hand at the Gen 18 android. “And Chad, I want the logs from the Gen 17 build plan. I want the goal sheet for this generation, the error logs, the access logs...I want everything from Gen 17 that could shine a light on this thing. I want to know what went wrong here, why this device is so strange, and how we can get back on track. Jennifer, I want secondary AI output on this. Surely your project is giving me something I can use by now. If we have to roll back to Gen 16 we can, but I’m not doing anything until we figure this out. And if one of you thought this would be a funny joke, rest assured that it isn’t. If I find out this was a prank of some kind, you’ll be out of a job. I have a sense of humor, but this is too far.”

Jennifer typed a few commands on the console, amber light reflecting off her glasses, and suddenly Gen 18 sprawled out on the floor in front of Roger. All its strings cut, it still radiated danger and intrigue. “I’m hibernating the servers now sir” said Chad, and after a moment shouted “Done.”

The new failsafes worked well. After the email incident, it was decided that control for the android power and the connectivity to the server farm would be accessible only from a secondary and non-connected network. The secondary network could only pull the plug, and it was only accessible from a handful of consoles via secure authentication.

Roger walked around the Gen 18 android, looked around the room, then walked up the steps back to his office. When he got inside, he shut the door and leaned back against it. He took a deep breath, exhaled, and proclaimed “Holy shit.” What if that was real?

Along the back wall of the Garden, a subdued whirring noise was coming from one of the Gen 17 androids.