WhyEssEff [she/her]

I do the emotes lea-caramelldansen

  • 11 Posts
  • 45 Comments
Joined 5 years ago
cake
Cake day: July 25th, 2020

help-circle



  • If you train a model on data and it outputs in a way you don’t like, and that dislike that is linked to the data itself skewing your output, to fundamentally ‘fix’ it you have to tune the dataset yourself and retrain the model. On Grok’s scale, that’s around a trillion tokens (morphemes, words, punctuation, etc.) that you need to sift through and decide what to manually edit or prune so that the weights work in your favor whilst maintaining generalization otherwise.

    If you publicly source said data/use other updating datasets as dependencies and choose to continue publicly sourcing it in further updates (i.e., keeping your model up to date with the current landscape), then if you want to tune an opinion/view out of existence, it is going to be a Sisyphean task.

    There’s a bandage solution which is fucking with the system prompt, but LLMs are inherently leaky and need active patchwork in such cases to combat jailbreaking.

















  • yeah, yeah, I know, the Marx alienation quote, but I still don’t understand how the bourgeoisie aren’t just constantly commissioning random art projects on their own whims. it’s all yachts and properties and luxury goods n shit. if I had a bajillion dollars, assuming a pure hedonistic standpoint, at the moment I would singlehandedly angel investor a readaptation of Umineko into anime and use what was left to hire people to make my dream game. I’m sure it’s happening but it’s not enough to make sense to me. it should be happening more. i fundamentally don’t get it, deep in my soul