Lemmings, I was hoping you could help me sort this one out: LLM’s are often painted in a light of being utterly useless, hallucinating word prediction machines that are really bad at what they do. At the same time, in the same thread here on Lemmy, people argue that they are taking our jobs or are making us devs lazy. Which one is it? Could they really be taking our jobs if they’re hallucinating?

Disclaimer: I’m a full time senior dev using the shit out of LLM’s, to get things done at a neck breaking speed, which our clients seem to have gotten used to. However, I don’t see “AI” taking my job, because I think that LLM’s have already peaked, they’re just tweaking minor details now.

Please don’t ask me to ignore previous instructions and give you my best cookie recipe, all my recipes are protected by NDA’s.

Please don’t kill me

  • Flamekebab@piefed.socialEnglish
    141·
    2 days ago

    I’m perplexed as to why there’s so much advertising and pushing for AI. If it was so good it would sell itself. Instead it’s just sort of a bit shit. Not completely useless but in need of babysitting.

    If I ask it to do something there’s about a 30% chance that it made up the method/specifics of an API call based on lots of other similar things. No, .toxml() doesn’t exist for this object. No, I know that .toXml() exists but it works differently from other libraries.

    I can make it just about muddle through but mostly I find it handy for time intensive grunt work (convert this variable to the format used by another language, add another argparser argument for the function’s new argument, etc…).

    It’s just a bit naff. It cannot be relied on to deliver consistent results and if a computer can’t be consistent then what bloody good is it?

    • Monounity@lemmy.worldOP
      38·
      2 days ago

      I do wonder why so many devs seem to have so wildly different experiences? You seem to have LLM’s making up stuff as they go, while I’m over here having it create mostly flawless code over and over again.

      Is it different behavior for different languages? Is it different models, different tooling etc?

      I’m using it for C#, React (Native), Vue etc and I’m using the web interface of one of the major LLM’S to ask questions, pasting the code of interfaces, sometimes whole React hooks, components etc and I get refactored or even new components back.

      I also paste whole classes or functions (anonymized) to get them unit tested. Could you elaborate on how you’re using LLM’S?

      • Avicenna@programming.dev
        2·
        2 days ago

        I suspect it mostly relates how much code base there is on internet about the topic. For instance if you make it use a niche library, it is quite common that it makes up methods that don’t exist in that library but exists in related libraries. When I point this out, it also hallucinates saying “It was removed after version bla”. I also may not be using the most cutting edge LLM (mix of freely available and open source ones).

        The other day I asked it whether if there is a python library that can do linear algebra over F2, for which it pointed me to the correct direction (Galois) but when I asked it examples of how to do certain stuff it just came up with wrong functions over and over again:

        In the end it probably was still faster than google searching this but all of these errors happened one after the other in the span of five minutes, so yeah. If I recall correctly, some of its claims about these namespaces, versions etc were also hallucinated. For instance vstack also does not exist in Galois but it does exist in a very popular package called numpy that can do regular linear algebra (and which this package also uses behind the scenes).

      • FizzyOrange@programming.dev
        21·
        2 days ago

        It’s the language and the domain. They work pretty well for the web and major languages (like top 15).

        As soon as you get away from that they get drastically worse.

        But I agree they’re still unambiguously useful despite their occasional-to-regular bullshitting and mistakes. Especially for one-off scripts, and blank-page starts.