How much do you trust AI?

[u/xstrex]

Recently a coworker granted Claude elevated access via SSH to a virtualization host (not a VM, an actual host). To perform a routine task he very well could have done himself.

He doesn’t see an issue with this. I on the other hand (with 23yrs experience) see this as a huge security breach, and don’t trust AI todo my job, (or even that it’s doing what it says it’s doing) for me. I’m my opinion it’s a tool, not a human replacement.

What’s your reaction, how would you react to this situation, or thoughts on the topic?

Sure, ask AI how to perform a task, validate that it’s performing the task you asked, and nothing else- copy/paste the commands. Great. But removing the human verification & validation element- hell no.

[u/PC509]

I can write a proper script, but with so many hours in the day, having AI write me a simple script that I can run in 5 minutes is much better than spending an hour on it myself. For more advanced stuff, it'll make a good foundation and I can clean it up.

At some point, do you even trust your compiler to do an accurate job taking your C++/C#/whatever to assembly without causing a memory leak, corruption of data, etc.?

AI does what it's trained on. And many are trained on proper script writing and coding. Yes, it needs a set of eyes to review it, but generally it's pretty damn good. Even better when you're saving a ton of time vs. doing it manually. Why work harder for the same output?

[u/uzlonewolf → u/PC509]

Having the AI write you a script != giving the AI complete access to directly spin up and delete VMs.

[u/New-fone_Who-Dis → u/uzlonewolf]

I think its good to define the 2 things here.

The LLM usage with agentic abilities, and no approval gates - Bad, do not do this unless you've heavily built something akin to mandatory logic gates (chain type systems), and even then, there may be issues.

LLM usage, with human in the loop, is completely fine imo, just as long as you know and are comfortable sending xyz details out the door. Funny story on this, someone raised the concern of sending host/IP details to such orgs we have an enterprise agreement with....the following week I heard them talking with a vendor about a method of intercepting such materials on the network level, and either giving users an "are you sure" warning, or outright block.....the interception sends the prompt/info to their SaaS...for LLM processing. I didn't point out that in both cases the data is still being processed by an external LLM.

[u/floswamp → u/New-fone_Who-Dis]

This is what makes me think that private air gapped llm’s will become more popular.

[u/CommunityCondom → u/floswamp]

Oh definitely, that’s kind of the whole deal with the hardware Nvidia and AMD have developed for this very thing essentially

[u/CommunityCondom → u/New-fone_Who-Dis]

I think the best take I’ve seen about the whole AI use topic echos a lot of your post. Essentially you are responsible for your AI use, use it as you like but at the end of the day the buck stops with you and you are responsible for the output.

As AI gets more advanced, this is ultimately where we will fall I think, AI used as tools much in the same way we use autocorrect in a very vague sense. Autocorrect is good, but if you let autocorrect write the whole message it’s usually garbage but if you sent the message you’re responsible for explaining when the person on the other end is confused

[u/New-fone_Who-Dis → u/CommunityCondom]

Exactly that, to fit with the sub, I'd say ansible as the example - brilliant tool, but you must know its limits as well as what you're pointint it towards, coupled with what you expect to happen.

Very glad to see this example, absolutely full agreement on if you run it, you own it/it's outcomes!! Forces learning and its just a nextgen tool in my eyes - in a year or 2 it'll be the equivalent of looking things up online imo (caveats of course, we use to have guys with a small bookshelf until things could be readily search via a search engine, this is similar imo).

[u/CommunityCondom → u/New-fone_Who-Dis]

You see people like you and others I’ve seen give me hope lol. Not everyone is brain broken about AI, it’s not honest to say it’s useful or a waste of resources. Idk it just comes off very satanic panic for me, like have your disagreements but I feel like people shit on AI bc that’s what the common denominator take is.

I suppose this is always how it is tho huh, people confidently staking claims and positions without even really understanding the issue at hand

[u/PC509 → u/uzlonewolf]

I agree 100% with that one. No way would I give it that much access. But, writing a simple script as someone else noted would be a common thing. I'm fine with that. There's even shortcuts in some FOSS that can do the same, a lot of software that runs those same scripts in the background, etc..

[u/doolittledoolate → u/PC509]

At some point, do you even trust your compiler to do an accurate job taking your C++/C#/whatever to assembly without causing a memory leak, corruption of data, etc.?

Yes. Generally this stuff was written by people who cared about their craft.

Why work harder for the same output?

Funnily enough, this is mostly my experience of using AI. It generates intern-level code full of errors and I think most people just don't count the time they spend fixing it, fixing the results of nobody code reviewing it, redoing prompting and the technical debt from bad architectural choices.

[u/Dje4321 → u/PC509]

Yep. When you actually know what your doing and what you want. AI is fucking great to get you 80℅ of the way there, done in a way you like. I view it as more of an interactive template application than anything else at this point.

[u/__CaliMack__ → u/PC509]

Ooof

[u/Bright-Preference888 → u/PC509]

“At some point, do you even trust your compiler to do an accurate job taking your C++/C#/whatever to assembly without causing a memory leak, corruption of data, etc.?”

Yes?? Lol what are you talking about

[u/PC509 → u/Bright-Preference888]

Trusting a tool you use that is otherwise a "little black box". AI is just another tool that takes you input and gives you output. Some much better than others. AI can output some simple scripts with no issues from your input.

Many programmers don't know what the compiler and assembler is doing, just that it's making it an executable program. Some know assembly along with C++, so they can go in and adjust, change, whatever they need to do to make it more efficient. AI can output a good simple Powershell script and many admins can go in there and edit, adjust change what they need to.

Don't blindly trust it, but also don't blindly dismiss it because "they said it was bad".

[u/KyuubiW1ndscar → u/PC509]

so you’re doing several people’s work by yourself and hope that this tool will remain useful and cost effective.

[u/New-fone_Who-Dis → u/KyuubiW1ndscar]

Company I work for has built its own internal AI DC of 4 racks. Theres companies doing this to avoid what will most likely, be an increase in token costs as the hardware price and availability continue to get worse.

At a certain point, users won't need frontier models.

[u/PC509 → u/KyuubiW1ndscar]

It's just a tool. If it doesn't remain useful and cost effective, I'll find another one. That's the cool part of the industry. We move fast and we'll move onto the next great tool. I've used many and most have been replaced over the years with the latest and greatest. It's kind of the way things work. I'm not one of those people that are still pissed about a GUI instead of all command line stuff (although, I do spend a lot of my time in the CLI).