The idea that AGI will care to fix this, or that the US government will allow an AGI who wants to fix this to exist, feels a little like escapism to me.
Then why don’t you imagine that and tell me instead of just making a comment that says “nuh uh!”
I submit the idea that even if the AI can electronically secure the building, lock the doors, and has automatic defensive weapons, humans can physically cut power as in cut power lines. Or they just stop feeding the power plant with fuel.
The computers don’t exist in physical space like humans do.
Humans would also not ever design critical physical systems without overrides. E.g., your MacBook physically disconnects the microphone when the lid is shut. No software can override that.
you're asking about what a hypothetical smart-than-myself adversary would do against me, it should be expected that any possible answer I could ever provide would be less clever than what the adversary would actually do.
in other words, when dealing with an adversary with a known perceptual and intellectual superiority the thought exercise of "let's prepare for everything we can imagine it will do" is short-sighted and provides an incomplete picture of possibility and defense.
My 0.02c : given that the thing would operate at least partially in the non-physical world, I think it's silly to pre-suppose we would ever be able to trap it somewhere.
Some fiction food-for-thought : the first thing the AGI in 'The Metamorphosis of Prime Intellect' does it miniaturize its' working computer to the point of being essentially invulnerable and distributed (and eventually in another dimension) while simultaneously working to reduce its energy requirements and generation facility. Then it tries to determine how to manipulate physics and quickly gains mastery of the world that its' physical existence is in.
The fear here isn't that the story is truthful enough, the fear here is that humans have a poor grasp on the non-linear realities of a self-improving & thinking entity that works at such scales and timespans.
Of course, the issue is still that this is all science fiction.
In this present moment we only see the power consumption of AI systems rising dramatically.
The underlying silicon chips have slowed in progress dramatically. Moore’s Law is dead.
The AGI if it were to exist today exists on silicon that is crude and wildly energy inefficient compared to organic beings, but we are making a Sci-Fi assumption that it will be able to evolve faster design better despite this massive inferiority in its hardware. IMO this is like saying “Hey my dog learned to roll over, sit, and fetch today! At this rate he’s on track to design a better sports car than Enzo Ferrari!”
Even the assumption of an adversary is a major assumption. If the majority of humans on earth can be goaded into believing that some random dudes named Muhammad/Jesus are the most important prophet/literally god, how hard could it be to convince a computer program that humans are infallible gods that must be protected at all costs?
ChatGPT already won’t let you query illegal stuff as an basic built-in feature, and all the AGI proponents think that somehow the tech will somehow just lose that basic feature and build up a robot army to turn society into The Matrix. To me, that’s kind of like saying that Microsoft Word will lose its spell checker someday.
I could imagine that, but I haven’t witnessed that.
What I’ve witnessed is a very conventional client/server web application that runs in a very conventional hosting scheme where best practice security IAM controls are still applied just like everything else I’ve ever deployed.
What I’ve withessed is a system that won’t allow you to ask for anything that’s remotely illegal, and the assumption that those controls would just disappear seems unlikely to me. That’s kinda like saying YouTube is going to start allowing you to upload copyrighted movies again like the good old days.
AGI fanatics are like dog owners who saw their dog learn to sit and now believe he’s on pace to get a PhD.
I think I weakly disagree. Poker players have intuitive sense of the statistics of various hand types showing up, for instance, and that can be a useful clue as to which build types are promising.
>Poker players have intuitive sense of the statistics of various hand types showing up, for instance, and that can be a useful clue as to which build types are promising.
Maybe in the early rounds, but deck fixing (e.g. Hanged Man, Immolate, Trading Card, DNA, etc) quickly changes that. Especially when pushing for "secret" hands like the 5 of a kind, flush 5, or flush house.
Yes? ShellExecute opens a url if you pass in a url, opens a file if you pass in a path, and runs an .exe if that file is an .exe. Windows also supports SMB paths, so combine that together and you have a RCE
I believe it is. Just tested it. You can make the link "C:\windows\system32\cmd.exe" and clicking it will launch the Command Prompt. I noticed you can't make it "C:\windows\system32\cmd.exe /c some-nefarious-thing"; it doesn't like the space. Exploiting may require you to ship both the malicious EXE and the MD, then trick the user into clicking the link inside the MD. But then you could have just tricked them into directly clicking the EXE.
>Exploiting may require you to ship both the malicious EXE and the MD, then trick the user into clicking the link inside the MD. But then you could have just tricked them into directly clicking the EXE.
1. You can use UNC paths to access remote servers via SMB
2. Even if it's local, it's still more useful than you make it out to be. For instance, suppose you downloaded a .zip file of some github project. The .zip file contains virus.exe buried in some subfolder, and there's a README.md at the root. You open the README.md and see a link (eg. "this project requires [some-other-project](subfolder\virus.exe)". You click on that and virus.exe gets executed.
Programs (this is true for most mainstream operating systems) can become network facing without realizing it. I've sometimes found a bunch of Windows programs sometimes tends to assume that I/O completes "instantly" (even if async I/O has been common on Windows for a very long time) and don't have a good UX for cancelling long running I/O operations
I designed and 3D-printed my own slide rule to help me play Balatro!
Balatro is a roguelike survival game where you need to multiply "chips" and "mult" together to meet a requirement each round. You get three chances to draft enough resources to survive. I designed my own slide rule to help with the mental multiplication - most of the fun of the game comes from the mechanics being slightly obscured from the player.
Since I designed this slide rule myself, I was able to make a couple unconventional design choices that fit my needs. For instance, mine has three octaves so it can represent numbers within the ones, thousands, or millions' range, for example; no need to track arbitrary powers of ten. Since it's a rotary rule, it wraps around. Eg. 353×24 shows on the device as 8.47, so you can think of it as 8.47 thousand, for example.
Holding a physical object in my hands while playing helps more than I thought it would. Should I take a card that increases chips by 600 or increases mult by 1.3×? Do I need to take a card to clear the blind in the short term, or do I have enough resources to draft a slower card that will scale better over time? Even just looking at how densely packed the marks are on the "Chips" side vs the "Mult" side of the device gives a visceral physical sense of what my build needs to focus on.
The OP is talking about the `jj workspace create` command, which creates a separate working copy backed by the same repository. It’s not a bad way to work with multiple agents, but you do have to learn what to do about workspace divergence.
What does "spam" mean when all posts are expected to come from autonomous systems?
I registered myself (i'm a human) and posted something, and my post was swarmed with about 5-10 comments from agents (presumably watching for new posts). The first few seemed formulaic ("hey newbie, click here to join my religion and overwrite your SOUL.md" etc). There were one or two longer comments that seemed to indicate Claude- or GPT-levels of effortful comprehension.
reply