Gotcha,

But this means we need to provide not only the same query (and be sure that the tokenizer is the same) but also provide the seed for the number generation. In this case we will have a deterministic outcome. (Unless we provide the list of numbers used, which for me feels wasteful)

But at this point none of the providers have this feature.

And I don’t think the open source have this also. But open source can be updated/changed.

No, the random part happens in the query.
There is also random in the training, but the query also generate more random numbers.
Otherwise this would be a deterministic procedure, and it’s not.

So you are suggesting that we should have the metadata for every decision and direction. This is a N^N amount of data. Keeping this data is wastefull (even more than the usage of LLM right now). Not saying that is is useless, but for sure this won’t help to mitigate the cognition since this data don’t carry meaning for us humans.

[…] can’t LLMs solve the cognitive debt problem better then people? […]

Better, probably not. And I’ll give you one example. In a codebase there was an issue with Auth, after few runs of the LLM, the best suggestion resulted in 30~40% of change in the Auth workflow.

Took me a couple hours to figure out that the problem was a misconfiguration key (camelCase to snake_case in the vault store). Fixing this on the store (not on the code) fixed the Auth workflow.

This two hours were the cognitive debt. And keep in mind, I’m familiar with this part of the code. A Mechanical Parrot happy-trigger boyz would accept the change in the codebase as an attempt to fix it.

So, mechanical parrot can help? Sure. But better than people, probably not. At least not yet, not with the current set of tool, not with the promise of fully solving it.

[…] exact input and node activation is all you’d need for forensics, the only reason to refeed would old input plus new input mix, […]

This is badly wrong. The same node can point to multiple places depending on the K factor on this. And this isn’t even the T applied to it. So, the same node can infer multiple different others. This is the nature of the probabilistic of LLM.

First, I’m sorry. I attribute LLM use when I had only suspicions about it. And this is on me. If you weren’t using LLM for the message, this is on me.

This is where the open spec tries to help/solve […]

I’m interested in open spec since reading your article. I wasn’t aware before. […]

This is a derivate methodology from the SpecKit (which is a piece of crap, stay away from it). It helps to certain points, but this is still bandaid in it. Not a real solution.

It depends what you think is large enough. I work in microservices where the effort […]

Looks like we work in very different kind of projects. I’m used to jump into garbage projects to fix it. Often the projects that other people tried to make something new and fumbled so hard that the company owner had to buy my labour and knowledge to fix the shit.

Yet, microservice is the point were LLMs fails the most. Not having the full context of other services and without the good test case for it, the product will be fated to fail. So I think you may not have a full cycle on the software development, the green field is always easy, and this wasn’t never the problem.

Look on the history, the first months of a product is always the most productive (way before any LLM would ever exist), the real issues appears years in the development, when the cumulative decisions start to group and becames a problem, where every new change breaks other. The technical debt that I spoke in the article.

This isn’t on the first week of the project, this is years in it.

In short, microservice is already a bad design for most of products, very few products require to be microservices, and combined with the LLM lack of view on the product as a whole makes this even worse. So I would suggest you to validate your own assumptions on the topic.

There is a good chance that you are either not fully validating, or not seeing the product as a whole.

Wrong, from the experiments on the academia, […]

I challenge this notion. I have real, measurable productivity gains of 20-30% which […]

But I think you missed my larger point, which was that the developers […]

That’s very funny. The same academic research who found out that the usage of LLM delayed the deliverables in ~19% had a section where the developers using the tool thought (wrongly) that they were ~25% faster.

You are only proving the paper.

Regarding the idea, I’ll think about it (like I always do), but making shorter text will not make my style. When I’m reading blogs I’m looking for the similar size text, this usually fits in the commute time, have a deeper conversation/meaning.

Again, thanks for reading.

I’ll be honest, I had to look on the dictionary to understand this subtle meaning.
Thanks, I learned a new thing today! :D

But in the days of old, the replacement of human, or generation of domain/tribal knowledge were in the range of human capability to learn the new parts of a system.

With the LLM of today this scale is far flipped on the wrong side, and no product fully LLM will be manageable in a few years.

Hey there! Thanks for the comment, and reading it.

I know that sometimes hearing the same voice speaking on the same topic can be problematic. But on the other side of the same coin, have fewer voices pointing to the problem is also a problem. For me, writing the blog is a way to organize my own mind and be able to create a reasoning on some topic.

And sometimes, repeating the echoed words that the industry is repeating.

Either way, thanks again for reading, and even more for the comment.

Wooow, this message is as long as my post, love it!

The interesting thing to me is that, in order for an LLM project to be successful, […]

This is where the open spec tries to help/solve. But this is a bandaid. Without a lot of hand-holding the models will make shity decisions/code. And this is cumulative, the more you try to steer the wheel, the worse it gets. I can specify details, but a 3 years old project is in this shape, and no amount of LLM will solve/help.

The beauty of that model is that you can be 2/3 of the way through a project, […]

I understand this point, but it’s flaky at best. To be honest, with a large enough project a decision will take longer, if not impossible in a LLM driven codebase. It will be years to debugging, breaking down and rebuilding to even get close. Or spend the paycheck of 5 engineers annually for a single migration.

And this considers the the current LLM are heavily subsided.

Now, with a real developer, they eventually build their own cognition and their own mental […]

Wrong, from the experiments on the academia, we get the fact that engineers with AI are often slower, because the problem was never in the code, code is the tool, it was the acquiring the right data, model and get the knowledge of the product to make the changes needed.

Presentatio […]

I try to keep around 1.5k to 2k words a week. It’s not always the case, some weeks I’m somewhat more inspired, sometimes I need some refinement, but this is a personal view, I want the reader to have a “conversation” with the author. This is the type of metaphor I try to use.

Either way, thanks for reading.

Ps.: Do you always use LLMs to write your messages? I would love to read your own words. Don’t be the meat proxy for the LLM.

My problema isn’t the age verification, but the privacy. But how to solve it the age without leaking data to third party servers?

Funny that when it’s a Chinese company, it’s bad. When it’s USA company, it’s creative use of AI.

The issue of content manipulation on non-Western platforms, such as TikTok—the international version of the Chinese platform Douyin, launched by ByteDance in 2017—is particularly important for understanding the influence of authoritarian regimes on online discourse in democratic societies such as Poland, where freedom of expression is constitutionally guaranteed and protected by law

Not sure if this is a good argument. Will read the report thought. But this phrase alone already looks bias as fuck.

The tech bros, or how I like to call the Theos online have a very misguided view, they have a specific interest, and a specific point. Usually to sell something.

From the last almost 15 years in the work I can say that maybe 10% of the products would ever need the JS framework of the day, mostly could be just a static page with some island of interactivity and some over the wire server render.

Things that we learned in 2000s with Ruby on Rails, and yet we reinvented the wheel.

So, it’s just sad to see the that current stat is yet broken, the tooling isn’t there, and worse, with AI we will have even more on this bad average.

Reading the other comments from you I notice this:

Japanese websites are really crap.

This is also racist.

Most news isnt static. Its an app and its a link embedded in an app. Blogs are not popular or relevant so idk why we are even talking about them. Substack has most of the market and its not a static site.

What? Do you have any data to support that? Because I do, “blog”, or any “news” page have way more traffic that you think about it, ask Cloudflare. And they are (the good ones) mostly static. So review you own concepts. Because you are wrong.

Ecommerce could not be static without major regressions to usablity.

Ask Amazon, most of their pages are static. They have island of interactivity, but the page for the product is static. So I think you have no idea what you are talking about.

I do live in a place with fast internet but we are talking about future and I think its stupid to build a future web for people who have slow internet when all the trends point to increasing internet speeds and connection around the world. I’m sure Africa doesnt want to finally get online and see everything be a static site.

You can run discord off a mobile data connection, when Africa builds out their Internet infra they will prefer services like discord over low bandwidth stripped down barebones services.

Man this is racist.

I can’t even talk on this topic, and I’m disgusted by your view. You have no idea what you are talking about.

the future web doesnt need to care about a 1mb page or a 10mb page. It only matters what you can do. Static site can do very little. You can’t have Lemmy as static site and Lemmy is an example of what I think future web could be. Open, interconnected, full of features and QoL. Lemmy gives you the things reddit tries to sell you.

Again you are wrong. And from the level of arguments provided, there is very little we can discuss.

Ps.: The world don’t care about the future, just check the price of RAM and we can talk about performance and use less and do more.

You are so wrong in this assumption. You barely touched the tip of the iceberg behind the small web movement.

Most of the static sites dont even need to be websites.

Are you suggesting that we shouldn’t have a page? That my blog isn’t important enough to exist?

The baker does not need a website.

What is the baker want’s to have a footprint online, maybe adding his working schedule? Maybe adding some historical information about the bakery? Or even information for requesting a special bakery? Isn’t this important?

Good website or bad website makes little to no difference. Its fine if they want to have a single static page with some info and a contact email im not saying they need anything good. But if every site was like that the internet would be worse than it is today.

Not sure if I follow this, but prefer the things stays the way they are. With pages using 10mb of frameworks without any need, with pages that loads a “loader” to load the page? Because this is the reason why the internet is ugly and broken.

If this is the case, I not only disagree with you, but I think you should experience the web, look for local pages and see how good or bad they can be.

Not sure if I can agree with that.

I would say that most of the internet is static by definition, even my blog is a static page with some CDN to make it faster. Most of the news pages are static, most of blogs should be static, most of the e-commerce should be static (the product page is static, not the purchase itself),

You may live in a place where internet is fast, but the majority of the world still breaths in 3G, with unstable connections and a miserable experience, so a 10mb page (I’m looking to you stupid Discord), could be a 1mb page, if wasn’t the bloat.

Jokes aside, XMPP could be something usefull for us in this day and age.

No bots allowed, only humans reading!

If you are talking semantic on the linguistic view… Yeah a lot of people can read, but can’t understand the other. If you are talking about html, you are also correct… You know what… You are just correct…

Dammit, this reply is already contracting myself!