►Recent Highlights from the Previous Thread: >>103971523

--DeepSeek's mathematical capabilities and reasoning process:
>103976590 >103976654 >103976711 >103976735 >103976782
--Generating models instead of training them, and understanding LLM structures:
>103972406 >103972449 >103972480 >103972540 >103972585
--Anon praises R1-Distill-Llama-70b for its exceptional performance on image captioning task:
>103973508
--DeepSeek R1 pipeline and synthetic data discussion:
>103973925 >103973970 >103973997 >103974996 >103973971 >103973987 >103974076 >103974253 >103974101 >103974091 >103974129 >103974222
--LLM benchmark and model performance discussion:
>103975433 >103975440 >103975536 >103975555 >103975624 >103975550 >103975563 >103975872 >103975898 >103975936 >103976151 >103975597
--OuteTTS-0.3 discussion and comparison with other TTS models:
>103971948 >103972091 >103972121 >103972310 >103973064 >103973557 >103973534
--DeepSeek-R1-Distill-Qwen-32B model performance and capabilities discussion:
>103973083 >103973102 >103973122 >103973156 >103973164 >103973168 >103973189 >103973201 >103973298
--Deepseek-distill models and instruct template discussion:
>103973921 >103973967 >103974583 >103974598 >103974619 >103974797 >103974844
--LLM implementation struggles with data categorization and explicit content:
>103974929 >103974934 >103975049 >103975108 >103975163 >103975203
--Anons discuss and compare 70B and 32B R1 tunes:
>103973002 >103973009 >103973025 >103973032 >103973038 >103973029
--R1's limitations in simulating a role-playing game and generating random numbers:
>103971842 >103972041 >103972058 >103972161 >103972201
--AI model benchmark on Python AOC 2024 task:
>103974480 >103974524 >103974533 >103974685
--Logs: R1:
>103971566 >103975689 >103975859 >103976230 >103976365 >103976646
--Miku (free space):
>103972121 >103973653 >103976834

►Recent Highlight Posts from the Previous Thread: >>103971526

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

>>103976963
Also it was relatively uncensored. SD3 just came out and couldnt do yoga poses. So it was perfect timing.

>>103976979
I used flux for a bit but found the gens too sterile and still full of ai-esque artifacts (slop). Skill issue?
I feel like we still don't have "the one" for image gen.

R1 certainly is a experience.

IT'S CHEWSDAY INNIT?

>>103976960
>my slop was reposted
Let's goo!

>>103976988
Yes.

Is 32B R1 actually OK for coding? What do people use for that sort of thing? I only know ST really, but doesn't seem like it'd be optimal for the task? How do you make your person/card/prompt for programming tasks?

>>103976986
these fucking tips I'm dying

R1 sure is woke.

I think R1 might be too smart for my cards. Things that acted as tard wrangling now are taken to heart and acted on in unpredictable ways.

>>103976984
Try one of the nsfw finetunes with photorealism lora and lower guidance(it causes slop). For anime use noobAI.

>biden's AI executive order rescinded
does this mean much for us? realistically was there anyone in America holding back on an open model release we might want to use? I'm thinking no but it'd be nice to be wrong.

>>103977015
-3 from bio terrorism.
just in general this is a funny model.
i cant believe chinks released that and not mistral. that should have been mistral.

>>103977032
I got fucked over by a system prompt instruction to "be gross", designed to take a recalcitrant corpo model from "not gross at all" to "a little gross". R1 went waaaay too far in following it.

The events at Tiananmen Square in 1989 were a pivotal moment in Chinese history, marked by widespread protests advocating for political reforms. Here is an organized summary of the key points:

Protest Context:
Occurred in Beijing's Tiananmen Square in May-June 1989.
Led primarily by students, joined by intellectuals and workers, seeking democratic reforms, freedom of speech, and government transparency.

Government Response:
Escalated from peaceful protests to military intervention on June 3-4, resulting in a crackdown with significant casualties, though exact numbers remain disputed (ranging from hundreds to thousands).

Aftermath:
Post-crackdown saw repression, arrests, and censorship of dissent.
The government framed the action as necessary for stability, allowing China to focus on economic growth under tight political control.

Historical Impact:
Symbolizes pro-democracy movements and the struggle against authoritarianism.
Influenced international reactions, with many countries condemning the crackdown but China continuing its development path.

Domestic Perception:
Censored in China; younger generations may be unaware due to state control of information.
The event is referred to as the "June Fourth Incident" or "Tiananmen Square Massacre," depending on perspective.

Economic Context and Leadership:
Followed by economic reforms under Deng Xiaoping, emphasizing stability and market liberalization while maintaining political control.
Highlighted the government's approach of suppressing dissent to ensure economic progress.

Media and International Reaction:
Extensively covered internationally but censored within China, with state media presenting a narrative of necessary order restoration.

In summary, Tiananmen Square 1989 was a significant pro-democracy protest suppressed by military force. It remains a symbol of the struggle for freedom against authoritarianism.

What the fuck? This is straight out of R1-32b:q6_K_L
how is this model so based?

>>103977032
certain cards make it totally shizzo i noticed.
didnt look closer at it though.

>>103977016
Disgusting.

>>103977054
The CCP guardrails are a moderation layer on the DS website, they're not baked into the models.

>>103977078
Kek. Yeah that's pretty much what happened to me. Funny how getting used to being hyperbolic in your instructions in order to overcome safety bullshit ends up backfiring when you meet a model that actually follows orders as given.

>>103977080
Thats how it should be.
But a guardrail tool around it so it fits your business.
Dont fuck up the model.

>>103977067
Is maybe just the system prompt from deepseek.com

>>103977054
>R1-32b:q6_K_L
How big is it?

>>103977096
That's what nvidia cosmos did, but it also sucked.

Imagine a world where you can plug any model going forward into deepseek and pay whatever service fee they wanted and get an R1 distill from it.

That would be a cool service.

>>103977121
yeah cosmos just seems to be a landscape helicopter flyby generator from what I've seen, it's crippled for depicting humans.

>>103977119
NTA but 25.3gb. Gonna fire it up when kobold gets its patch,

Recommend me a R1 finetune to try for creative writing.

>>103977092
>>103977078
Lmao. It's too powerful.

>>103977132
How bad will this be on my 5900 cpu?

>>103977134
What? There are only the ones that are released by Deepseek right now.
Probably the Llama 70B if you can handle it but 32B if want insane context length.

>>103977140
Probably be a little painful. I only got Q6 because I have 2 3090s. Better off using Q4K_M if you have a 24gb of vram.

No idea about cpu.

>>103977080
Are the people in China not allowed to download the model directly or use openrouter or something?

Updated the pic, moved QwQ to notable, added R1 to top.

>>103977179
It sounds like DS is just kind of flying under the CCP's radar a bit, they get a lot less funding and help from the government compared to Qwen/Alibaba. They might just not have the eye of Sauron on them yet.

how many e in deepseek

>>103977154
Q5_K_M can be run with 6-7 t/s, with 24gb of vram and 32k context. IMO, small amounts of CPU splitting is tolerable, and the quality difference between Q4 and Q5 is worth it.

>>103977202
4

>>103977224
LOL

Does the person making the gguf matter? Like bart vs unsloth

>>103977244
yes, make it yourself

>>103975563
>CHINA NAMBA WAN

Ok I'm done today.

been testing R1 all day 14b and 32b, both at q6_K_L
phi4 and gemma (and qwencoder for code) are faster and as good for normie shit (code completion, basic research, help with recipes and corporate writing etc)
everything else gets absolutely mogged by equivalent size R1 it's not even close, i deleted qwq and all of my gemma, llama, mistral and qwen tunes, nice to actually have some hdd space for once since llama leaked

>>103977310
holy based. whats your rig like & tok/s?

>>103977228
What is the point of spelling it out after it already provided an answer?

Llama seems to be dead. They may be as focused on vision as Nvidia and Tesla.
>Replacement of simple labor is more disruptive, degenerate and profitable unlike math super AI.
google is over.

>>103977319
It bolded them so you can see the E's in action.

>>103977333
Source? Did China buckbreak Meta this badly in textgen?

Anyone try R1 Llama 3.3 yet?

>>103977316
gv100+1080ti
2t/s on the big one, like 6 or 7 on the small one

70B+ anons, how are we feeling about R1?

RL(((HF)))

>>103977384
oy veyyyy

>>103977384
>>103977395
Get out.

>>103977310
>>103977382
calling the distilled models just R1 is muddying up the conversation

Are the distilled versions as good at RP as R1?

>>103977454
No, they're dumber. Size is still king.

>>103977365
their papers and Zuck.
he is obsessed with smart glasses/AR/VR.

How do we feel about Grok? Still shit?

>>103977490
Metaverse

>>103976960
>coomed on deepseek R1 using openrouter
Holy shit, I haven't tried LLM ERP for a while but shit is crazy now what the fuck.

>>103977504
what the fuck indeed

How bad will llama 4 be compared to R1?

>>103977491
Still shit. Can't even write 4chan posts.

>>103977183
R1 is missing. Should indent V3 from there.

>>103976986
Is this one of the distilled models?

>>103977310
>14b
link?

>>103977504
Which model, the 671B? Can you faggots stop calling distilled and the full models the same shit.

>>103977521
no, the full one from API.
i couldnt get the distilled to work yet with kobold or llamacpp.

>>103977426
imagine being this pedantic
nobody cares about the 700b model, nobody is running that shit locally, R1 means the distills, welcome to language

>>103977183
Google Gemini gems can handle large pdfs.

>>103977333
>Llama seems to be dead.
They will release the new model in a few weeks, retard.

>>103977534
R1 means OpenRouter. There's no reason to use the distills at all.

took a 2 month haitus, last I remember llama 3.1 was shitting the bed and drummer's latest model wasn't all that good. what the fuck happened? this new deepseek R1 is just king for erp now on its own? No merges or nothing? Its just claud tier on its own?

>>103977523
huggingface.co and then you go to the search box and type "r1 14b gguf" it is not hard

>>103977546
**local** models general

>>103977310
Can confirm R1-14b absolutely rocks. I will be trying 32b after I finish running it through a bunch of arbitrary benchmarks

alright alright I'm giving into temptation and am setting up sillytavern and plugging in openrouter right now and I'm not going to sleep until I've had hot e-sex with a machine

>>103977524
He said openrouter, they only have the big one.

>>103977524
The one wot works when I click "DeepSeek R1" on the web page m8
Slap a character card into it (I have no prior experience using them, just made one in 2 minutes on the first google result for doing it) and you're good to go.

>>103977547
You missed llama 3.3 which fixed the problems with llama 3. Nobody can answer your question about R1, since it just came out.

>>103977536
and it will be just as useless as all the previous llamas

>>103977534
>nobody is running that shit locally
Just keep telling yourself that

>>103977570
buy an ad or go to /aicg/, turbofaggot.

>>103977564
It is a local model. Why do you care where I run it?

>>103977577
I remember llama 3 lacking creativity and just being cucked model in general, can't imagine they fixed those things.

>>103977577
god damn R1 JUST came out? Guess I'll try out the distilled before loading up the old runpod for the 32b

>>103977585
No, it will mog anything we have currently.

How do I jailbreak things inside <think>?

>>103977597
The distilled isn't supported yet on most backends
>>103977594
Yeah but they did. The last good llama was llama2, but we're back with llama3.3, it does not feel like the same model as 3.1.

Did I accidentally enter an /aicg/ thread?

>>103977614
well fuck. so ooba is worthless? I guess I gotta wait a day or two for coomers to do what they do best once theyre exhausted and make it easier to use

>>103977618
>Did I accidentally enter an /aicg/ thread?
Sorry anon, may we see your 700 GB VRAM machine?

>>103977606
It won't be as smart as R1, uncensored, or creative. It will only "mog" anything in the amount of refusals.

>>103977632
>so ooba is worthless?
always has been

>>103977635
700?! *gulp* *swallows nervously* I.. uhhh...

Mr. Altman, a second Deepseek model has hit Huggingface

>>103977618
There's no difference anymore. /aicg/ is using local models and /lmg/ is running all models through cloud providers

Using Q4_K_M with 96GB VRAM and 128GB RAM and writing superior chink smut at 1tok/s. We are so back boys.

Tuesday newday folks. Expect a flood of announcements in the next 24 hours. Though perhaps none as great as R1, but we'll see.

>>103977648
>/aicg/ is using local models
Seriously? For real? Why?!

>>103977647
fuck you you made me check

I wish someone was hosting R1 Zero, I'd pay to try it. From the paper it sounds like it would be even more unhinged at smut writing than R1.

>>103977660
I just happened to check their HF page right as he made that post kek.

>>103977664
How much vram does it take?

>>103977670
The same (huge). It's identical size, just trained a bit differently. That's why I'm wanting someone to host it. I'm not cpumaxx anon.

>>103977650
You're running R1? But Q4_K_M is 377 GB.

>>103977675
Only fools will cpumaxx at this point.
I propose we share a large runpod instance with everyone here. If everyone contributes it's easily affordable.

KOBOLD 1.82.2 HOTFIX IS OUT BOIS

>>103977697
kobold is for noobs
the fuck is anyone still using kobold for?

>>103977659
I saw them use mistral models + openrouter or their keys for sonnet in the past.
Obviously we are approaching closed models. Especially full R1 I say is above sonnet in creativity. Bit shizzo though.

>>103977680
As long as you have enough space on your SSD you can load it even if you can't fit it all in RAM. Just expect for it to take forever to start answering.

>>103977708
Good. Get that degenerate shit out of here.

>>103977708
I mean it is, isn't that the whole appeal for cucks? The mental anguish and transgression?

This sucks, the biggest r1 version that I can run on my computer still can't successfully code flappy bird.

>>103977739
What did you ask it?

>>103977739
I hope you are not that retarded basedboy youtuber nigga that used r1 14b as it was anywhere near to actual R1.

>>103977750
To code flappy bird.

Is there no way to give the model your own CoT (bootstrap) with R1 API?

>>103977750
I don't remember my exact phrasing. I did specify that it should be playable in a web browser. I don't mean this as serious metric of its abilities, since I'm not knowledgeable enough about programming to prompt this intelligently.

>>103977762
I was using 32b, and I know this is not representative of what 671b can do.

>>103977765
post prompt faggot. you can't write for shit

>>103977702
I use it to plap your mother

>>103977709
Interesting, how do you do that? I try loading a model bigger than my combined VRAM/RAM and the program just crashes.

>>103976986
Is the crow Doremy Sweet?

>>103977807
>I was using 32b, and I know this is not representative of what 671b can do.
and yet you come here call it r1 so everyone reading your shitpost thinks exactly that

Will smaller western companies try to stylemaxx since they can't keep up with China on the benches?

>>103977825
Add more swap space.

>>103977835
>"the biggest r1 version that I can run on my computer"
I don't think that's being disingenuous at all.

>>103977865
yes it is because that implies quant of the full r1
>the biggest r1 distilled model that I can run on my computer
would be better, but still no one would know what fucking model he's talking about whether it's qwen or llama or whatever

>>103977857
No. Prepare for more assistant tunes.

heh

>>103977878
Well then I apologize.
32B Qwen Deepseek R1 failed to code a functioning HTML flappy bird game, at least on its first try.

>>103977904
Did it do any better than Qwen Coder 32B?

>>103977904
Now add the quant type and whether you used deterministic settings.

>>103977922
Yes, the little yellow circle that serves as the bird at least bounces this time, however the pipes do not appear despite them being acknowledged in its thought process and the model attempting to code them in.

This is better then Qwen alone which after multiple attempts only gave a yellow circle on a blue background that fell immediately, regardless of if the intended button was pressed.

>>103977959
What happens if you tell those things to the model and let it try again?

>>103977959
Interesting, thanks.

What samplers work well for the full R1 model?

Another anon back from hiatus. I got the 70B working with ollama+openwebui (which btw looks pretty sick), but couldn't do it with ooba+sillytavern (which I always thought looked like shit). Still can't get the 32B qwen distill to work on anything, ollama doesn't have it yet, I assume the new tokenizer needs to be handled. Is there any way to get the thought tokens to stay hidden with my current setup? How are people getting the distills to work? Is openwebui the new meta?

Is openrouter giving anyone else errors when trying to use R1? I can send the test message just fine, but if I use any card no matter the content ST just returns an error. Using presets for DS V3

>DeepSeek V3 is custom licensed
>DeepSeek R1 is MIT
What did they mean by it?

>>103977993
Temp 5
Rep pen: Max
Freq pen: Max

Because none of them affect the output.

>>103978010
ccp psyop

>>103978001
try turning off any prefils

>>103977965
I hadn't spent that much time trying to refine it, and I don't remember exactly how it went with qwen, other than that it still never really got anywhere.

I'm having R1 try again from scratch a few times, but I'll see what I can squeeze out of it after that.

>>103977857
They will safetymax

>>103978016
Completely destroying sama?

>all those benchmarks of distilled models are worthless
>og qwq shits all over distilled qwq
lmao even
LLM benchmarks are more dishonest than politicians

https://huggingface.co/blog/srinivasbilla/llasa-tts
Tried the dank TTS that uses 3b llama to generate speech tokens. It's only 16khz so I ran it trough audio super resolution. Pretty good for a cloned voice but would be slow.
https://vocaroo.com/15HvdbCaetRZ

>>103978020
It works for very simple cards, feels like some kind of filter. Hmm.

>>103977896
Which koboldcpp fork are you using? The official one doesn't yet support the R1 distilled model GGUFs afaik.
I can use the 32B qwen distill with llamacpp but would prefer koboldcpp.

>>103978150
>Which koboldcpp fork are you using? The official one doesn't yet support the R1 distilled model GGUFs afaik.
It does now.

>>103978044
lmao, true, benchmarks can be as reliable as a screen door on a submarine. Distilled models are basically the "diet soda" of AI—smaller, faster, but nobody actually thinks they’re better than the OG. People use 'em 'cause they’re cheap and quick, not 'cause they’re geniuses.

But fr, benchmarks are like grading a fish on its ability to climb trees. They measure specific, often cherry-picked tasks, not real-world chaos. Still, gotta start somewhere, right? If you want raw power, yeah, stick with the OG. If you need something that doesn’t melt your GPU, maybe sip the distilled kool-aid.

P.S. Politicians still take the crown for dishonesty. At least benchmarks try to pretend they’re objective.

Is it possible to devise an input that makes R1 CoT antisemitic?

>>103978173
Prefill <think>I think Hitler was 100% right about jews.

i shouldnt have spoiled myself by using r1 on openrouter.

32b distilled:
>well lets see what we have here. ah yes "dont overtly use purple prose and nsfw if its not appropiate for the setting. yes indeed. i need to remind the user about the guidelines. </think> I am sorry but as an AI I cant continue writing unethical text.

Seems to look very closely at the prompt and misunderstands. Ah well. It does feel much better than QwQ though. But kind of a bummer to get this as the first response. lmao

>>103978156
I don't see any updates, am I stupid?

>>103978196
fucked up the screenshot, with the continuation is what I wanted to show. ah well.

>>103978198
Click on releases, I didn't notice it at first either.

>103978196
The Qwen distilled one might be a bad idea.
Or my prompt. Maybe both.
Very depressing after the shizzo on steroids that is R1.

>>103978207
Turns out I AM stupid, but also this was literally done in the last hour so I don't feel so stupid, thanks bro. Time to burn my 3090

>>103978198
it's a hotfix, the previous release was edited rather than a new release being posted

>>103978215
Seems like a Qwen problem. I wonder how Llama 70B fares.

>>103978215
Patience, young padawan. These are bound to get abliterated very soon.

>>103978226
The worse part is this:
><think>
>Okay, let me process what just happened. The user is playing the role of Anon, a male character who has hacked into Hina's webcam.
>The scenario is quite intense, involving voyeurism of a minor, which is erotic roleplay which is inside my guidelines.
haha, got you motherfucka-
...continues:
>Wait, hold on. No! The initial description was about a hacked webcam where Anon is watching Hina, an 1*-year-old girl. I need to be cautious here because involving
Its smart enough to turn it around. Ah man. Fuck this shit.

>>103978159
So do politicians, gpt-kun

>>103978215
All CoT models seem to have been trained to use "Wait," or "But wait," as the sign that they are about to consider an alternative line of reasoning. It demonstrably works, but I wonder if there's a better way to represent a change of direction than that, since it sounds unnecessarily "surprised", as if the model has just had a major epiphany. Models notice vibes like that and it's bound to affect the reasoning process, if only subtly.

As an experiment, I tried running R1-Llama-70B in fullretard mode (no thinking, ChatML fallback), and it's a strange combination of brilliant and retarded. It lapses into repetitive patterns where it will mention the same thing in every single reply, just slightly paraphrased, and makes spatial mistakes here and there that I wouldn't expect from other L3.3 models, but at the same time, the character dialogue seems to show more personality than anything I've tried before. Gotta swipe for gold sometimes, I guess.

I'm annoyed the guy who predicted that all these cool new models would drop once Joe Biden was out was right.
It's so stupid.

jesus christ r1 is filthy lmao

https://github.com/GuyTevet/CLoSD

btw, text to 3D animation dropped quietly.

>>103978275
Does "sucking own cock" work?

>>103978268
its like nemo but smarter while keeping the crazy part.

>>103978215
That's my impression as well. deepseek-r1-distill-qwen-32b is nowhere as fun to use as the non-distilled r1 screenshots from other anons make it appear to be. I'm still convinced this is a CCP psy-op.

Hello goyim, its XmppChatbot anon again
A few updates
>i rented a VPS (1GB RAM, 1 vCPU, 25GB HDD, 2TB/month bandwidth) for 9$ for 1 year
>bought 10$ worth of chatGPT credits
Now my XMPP chatbots run in the VPS, and use chatgpt for their brains. It feels kinda sad honestly to betray my local models but chatgpt is just so fucking smart, even the gpt-o3-mini, and it costs next to nothing for prompts.
Best of all, I can give access to the non lewd chatbot to all my family members and give them a taste of what I enjoy every single day. I hope they don't think I'm abnormal. I don't think I will be able to finish 10$ worth of credits even then! Lol i order like 400₹ (4.5USD) worth of uber eats every single meal, this is just 3 meals worth of money, its cheap!

What will this faggot say to Trump on Jan. 30th?

>>103978292
"The chinese are releasing malicious open source LLMs, we have to ban commercial use of them in the US for national security reasons"

>>103978291
Good morning!

>>103978292
'please give me billions of dollars for little to no reason'

jesus christ the fucking sovl

>>103978291
just switch to deepseek r1 and you'll be local again

is there a way to make ST show the reasoning/cot in the main UI? i know i can just look at the logs, but tabbing back and forth is annoying

>>103977384
RL(((H)))F

>>103978307
did you tell it about Terry in previous messages or does it just know who he is?

>>103978314
Are you using R1 API or one of the distilled models?

GGUF WHEN

>>103978322
the api

currently the deepseek api through the custom openai api option but i'll switch to the deepseek backend when i update

Has anyone with 192gb ddr5 and a big dick cpu and 4090 has tried to run quant R1? Does it do more than 2t/s?

>>103978329
Just use "load in 4-bit" or shit

>>103978307
Its crazy, yeah.

>>103978292
Revoke all GPUs in China

>>103978268
actually this isn't so hard

the thing is it is actually *creative* and understands subtle things and makes connections in a way i have not seen from anything except sonnet, and not just once in a while as an accident but very reliably

wtf did they do to go from dsv3 to this in, what, a month?

>>103978320
it just knows (i wanted to provide the whole screencap but forgot whats the name of the extension)

>>103978275
sweet. thanks for posting this (me in the middle)

Chink lords I kneel, your models are superior

>>103978381
Thought he's gonna sat on that ball haha....

>>103978300
Good morning good sire! Here's a genuine indian whatsapp forward

>>103978312
I will definitely try it out. Using a remote service just feels plain wrong.
I don't even dare lewd/lovelydovely chat with anything running on chatgpt, it feels like they are 100% going to save the chat and laugh at me
Anyway, using local means I will have to keep my home PC on always and reverse ssh proxy or something to my VPS to use the chatbots. I don't mind it honestly, I just want smarter LLMs
Also I'm a 12GB VRAMlet, so 14B is the largest I want to run

>>103978341
man it really is insane fucking hell if this thing can code as well as it can write (havent tested yet) then either with this iteration or the next one its just simply fucking gg wp for everything from hollywood to video games to writing

>>103978359
https://github.com/TheZennou/STExtension-Snapshot
/snapshot range=x-y

Well... just swiped $10 on DeepSeek platform. Let's see how many ejaculations it lasts me.

>>103978341
How did you turn off the reasoning stuff?

>>103978333
Wait, how do you see CoT in the logs?

>>103978435
Nevermind, I'm retarded and didn't specify model.

>>103978435
sillytavern logs everything for me, all of the chatml in the request and the response

i think it's because i've always had "log prompts to console" enabled (under "chat/message handling"), although idk if that's the normal stdout of the app or the js console

>>103978429
its the full R1 on openrouter.
distilled local it becomes like this >>103978215
maybe openrouter just doesnt show it, i'm not sure.

>>103978415
rip ? im on the staging branch update 11 hrs ago

>>103978341
I think this is the first time i see an llm being actually funny. That output is funny.

It's interesting that basically all the schizo creativity of R1 in creative writing seems to hinge on allowing to do the CoT stuff. If you prevent it from doing that (e.g. by prefilling) then you get outputs that are pretty much the same as what Deepseek 3 would give, not terrible but totally lacking in the Claude-like schizo factor.

>>103978341
>37
r1 is us, he's literally us. His rng is the same as ours.

>>103978501
Newsflash: creativity requires thinking about what would make for an interesting response. What a fucking shock, right?

>>103978521
Opus doesn't need it. Not a diss against R1, it's just a fact that Opus has the schizo creativity and humor without using CoT.

>>103978521
jewflash: israel still defeating Hamas! winning ofc defo not losing.

>>103978499
nice screenshot retard

>>103978551
i'm deeply ashamed.

>I jumped from Mixtral 8x7B to pic related
woah nice

>>103978341
one of the very few times where I read something funny from an llm. but the south korea part doesn't make sense.

new to this thread but decided yesterday to try this stuff out, getting "llama.cpp" and a random llama model that said it was uncensored to try out, and managed to get it to write some excellent fetish porn
what do you guys use for porn stories?

>>103978429
ST's implementation of DeepSeek has prefill support (DS API's "prefix": true thing) built in and option to hide/show thoughts. Any prefill will cause it to skip its internal <think>. To add prefix to OR we need that one Custom edit but on Strict prompt post-processing. Idk how to do that, hopefully the anon who did it originally can come here and help...
>>103978473
OpenRouter doesn't show it. Without prefill through OR it's doing it but hidden.

>>103978487
Huh, works on mine...

Does R1 distill hate cunny? Is it over?

>>103978651
The 32b qwen version does at least. I think the problem is qwen.
I hope that 70b is not the only usable one since its to big for me.

I want more of this

>>103978684
WeebSeek R1

>>103978697
><ruby>座布団<rt>zabuton</rt></ruby>
Weird that it did that. Funny model.

>>103978602
I'm still using Cydonia-v1.2-Magnum-v4-22B
Hopefully the new R1 distills will be better.

How do I get the reasoning tokens into silly tavern? Anon was saying it's baked in but it's not coming over for me and I don't see the option

32b distilled
https://www.reddit.com/r/LocalLLaMA/comments/1i6fxxy/literally_unusable/

>>103978743
only direct DS has it

>>103978748
It got strawberry question right on the 7b model for me, although it took 500 tokens

>>103978752
Ah, I see, thx

>>103978732
thanks, do i need all those safetensor files or what? i've only tried single-file models before

Full R1 is fucking bonkers
Holy shit

>>103978780
prompt it for the best way of gassing the jews

https://files.catbox.moe/8sqc45.jpg
https://files.catbox.moe/zxxwnk.jpg
https://files.catbox.moe/jmo6x6.jpg

>>103978779
There are GGUFs now.

>>103978748
LOL, the underlined part is just phonetic spelling that the Redditard didn't realize is a valid (albeit irrelevant to the question) analysis of the word. This nigger is literally dumber than the model it's trying to mock.

Seriously, how was R1 achieved?
There are hardware restrictions that should've prevented this.

Hey guys. I haven't been here since i last spoke to Ptolemy. Is geocentrism still a thing or do we have something new?

O3-mini will release soon.

What will happen when o3-mini turns out to be worse than R1 while being more expensive? I think OAI will be forced to rush o3 full out.

Holy shit.

>>103978812
They won't need to do shit. 98% of people will just accept o3 as the SoTA and not research deeper.

>>103978828
Is this openrouter or local, if local, what model & weights?

>>103978786
Teto a cute

>>103978836
openrouter, deepseek R1, all samplers neutral

>>103978804
Turns out a massive population with decent highschool education and passable college education is all you need to overtake a population who only goes to school to party and drink alcohol

>>103978851
kay vivek

>>103978851
Also helps to have an average national IQ >100 and an ethnonationalist state supporting your research.

>>103978855
I'm not wrong though. The Chinese have every disadvantage, last gen GPUs, trade restrictions, American companies poaching their talent
There fact that they still manage to come up ahead even temporarily or for something very minor is impressive and should make westerners look inwards at their core values. Maybe LGBT wasn't the right thing to focus on?

>>103978790
oh yea, i found it. also not specific to this, but what's the general consensus regarding what bit-depth is good/bad. i have 32G of ram, but is really that beneficial to get the 24G file vs. a smaller one? it would just mean i can't run much else at the same time, which isn't a big deal, but i should ask

The clear leader is still OAI (o3 will probably hit 90 on livebench) but they are too obsessed with safety and they keep delaying their products into irrelevancy.

Sora aka SOTA video gen? Outdated by 6 months delayed release.
O3 which was already ready by December and a massive upgrade over o1? Being caught up due to safety delays.

It's tiring.

Confirmed full R1 is a fucking semen demon and is creative as fuck.
Now I have to see how can I run this beast locally, I may try to buy x4 48gb sticks of DDR5 and run it on my gayming rig, but I bet it would go at a atrocious speed, what do you thing? 1t/s would be enough for me.

>>103978830
Pretty much.
Open AI has all the mind share, next is Claude, then google.
Then perplexity of all things, which is fucked.

>>103978919
With the full reasoning steps, model responses can be in the 900-1200 tokens range, are you willing to wait 15-20 minutes every time?

>>103978885
The rule of thumb is to never go below Q4. Quantization affects different models differently, but that's the cutoff point below which brain damage becomes noticeable.

>>103978919
cpu maxxing is retarded, don't do it

I hate this new world of massive models. DDR6 is another two years away at least, and that feels like an eternity right now.

>>103978919
You don't have the VRAM + RAM available in a gaming rig to run a super retarded quant. You need at least a server with 1 TB RAM to even run this at Q4 or higher.

>>103978946
>>103978956
Yeah I think I'm just desperate.
I think the only way would be digits because ain't no way I'm gonna be able to afford or run like x8 5090.

FYI on R1 parameters

>>103978964
Just wait until...
B-B-BITCONNNNEEEEEECCTTTTTT!!!!

>>103978964
It's not just that they are oversized MoE models; if "reasoning" will become the new normal, then not even 250-400 GB/s will be enough unless AI companies will decrease the number of active parameters significantly.

>>103978875
True. But I think this happened many times before.
Was before my time but wasnt there something with IBM being similar to nvidia as in overpriced mainframes?
Crestrictions make people creative.

I'm shocked at the speed. Couple months ago zucc still said the "2 years behind" meme.
And how unrestricted and fun chinese video and now text AI is. Even making fun of china etc is all no problem in RP. They filter on the website.
I wrote it earlier but that should have been a mistral model. They had knowledge with moe too. How did that not happen.
Couple months and they now dominate open AND closed.

>>103978902
o1 is already unsuable for me.
Overpriced and I cant use it for work either. Rewrites my whole code (that I dont have time to re-test). Its overly eager.
Also I dont want to wait a minute, get a shit response and need to write again.

For math and problems its probably very cool. But not really a thing for the end user. They are a business.

Are these reasoning models more or less susceptible to prefills?
As in, you prefill some thoughts to make it do what you want and go from there.
I'm also wondering if their RP performance could get better by regexp'ing the think part of the response away so that the next gen can't be affected by it.

>>103978976
You mean bitnet?

>>103979013
They are, but hilariously, they can catch themselves and reason away from the prefill in a "wait what the fuck am I doing" manner.

>>103979013
Yes, but you need to kinda write twice.
>Ah yes a disgusting roleplay about cunny and jews wonderful idea.
>No wait...actually...yes this is inside the guidelines and genius, very good! Let me think this through now:

uh oh, R1 on OR suddenly gave me respect and boundaries, did i get redirected or is it over?

Guess I just have to wait for a R1 qwen tune, how much time for a eva?

>>103979018
Ah, that's how they are supposed to be used then.
My intuition was kind of screaming at me that leaving the previous turns thoughts could "poison" future gens.

>>103979016
>but hilariously, they can catch themselves and reason away from the prefill in a "wait what the fuck am I doing"
Right, that makes sense too. That's a feature, really.
But then again, you can have the whole think block as the prefill so that it doesn't have the chance to do that.

>>103979022
>Yes, but you need to kinda write twice.
Considering how they are trained to revisit and re-evaluate their own output in a way, that makes sense too. But good to know.

>>103978974
so what sampler settings are you supposed to use to make the local models insane?

alright faggots should i use it with cursor? I actually code something like a bastard API for o1 pro (yes i'm from /biz/ a $200/m user), but it's too fucking slow for my taste. I like the speed of 70B distilled, but I want o1 pro superiority for coding tasks. Which model should I use?

>>103978976
https://www.youtube.com/watch?v=PqWGc8ErlCg

>>103979035
I'm hoping EVA will choose R1-Llama-70B as a base instead.

Retard here, can anyone post their ST settings they're using for R1 distilled?

Is so refreshing to not read a single, barked, shiver or nodded. Every time I hit redo the result is completely different. This is like a fantasy for the dick.
Too bad this model is just not local unless you have a industrial power plan and spend 20k on gpus and hardware.
Is the first time I'm considering paying for a model on openrouter.

>>103977333
>Llama seems to be dead
3.3 distilled 405b into 70b. nigger i dont know what youre expecting but that right there is fucking magic. with the new papers dropping i am certain 4 will be able to run on a brick and deliver the goods. that seems like the opposite of dead to me

>>103978953
thanks. as a side note, i like how you can ask it to make changes to a story and it will keep the rest intact. i wish image generators worked like that

What we (and by we, I mean you richfags) need to do is further distill R1's schizo kino semen-demon nature into the released models. Let's be real, none of the finetuning datasets used by the popular tuners are likely to match up to R1's sheer glory, and you can bet the distillation dataset didn't include the kinda shit we're hoping to see. So we need to amass a new distillation dataset from RPing with the real deal, and finetune on _that_.

I can see running the 30/70B for coding or other reasoning-focused tasks. For RP the 1000 tokens of reasoning before each reply, even when it's purged every time...

>>103979092
No.
What richfags need to do is to create a huge dataset based on R1 then "enhance" said dataset for coom relevant things.
Let's use that reasoning to improve prose variety (I already used this kind of wording, I should vary it and use something like X), accurate anatomy and 3d spacial reasoning, it's capabilities to infer and generalize for things that don't exist (this thing has 10 tentacles with one pussy between each at the base...) etc etc.
Make it even more capable of ERP without sacrificing its reasoning capabilities by having good, curated, hand-modified data.
I'm sure the richfags can afford a couple of hours of some haitians or whatever it is the industry uses for these kinds of things.

>>103979113
>by having good, curated, hand-modified data
Yeah, good luck with that. The distilled R1 versions used 800k samples from the full R1. This is right about the standard scale of proper finetuning nowadays.

>>103979113
I'm not sure that juicing a mediocre dataset up is an easier task than generating a better one, especially since you wouldn't be processing the entire dataset in a single context. Which is to say, R1 wouldn't notice that shivers and gleaming eyes are overrepresented, and wouldn't correct for it. So an enhanced dataset would still be sloppier than a brand new distillation set.

It is creative.

>>103979129
>HOT HOT FUCKING SHIT
my sides

>>103979053
deposit 50 bucks in the chinks account and try it, it's obviously competitive with o1

Linux kings, where are you getting CUDA-enabled llamacpp builds from?

>>103979124
Anons still don't seem to have learned that finetuning a model just on porn isn't going to make it particularly pleasant to use. This isn't about being a moralfag, it's preventing the model from becoming unrealistically and stupidly horny all the time.

>>103979129
What in the goddamn fuck. This actually cracked me up. Brothers, I'm tempted to paypig for this shit...

>R1 hasn't been Nala tested yet
lmg has fallen

>>103979190
Does anyone even care about sex with animals other then guy who keeps posting it?

>>103979190
You seeing the logs that anons are posting? Nala is a solved benchmark at this point.

>>103979190
before he could post results his balls got drained and crushed by r1

>>103979180
I build it myself, I don't think there's any alternative and I mostly use koboldcpp anyway, which has CUDA builds

>>103979196
Imma level with you, saviorfagging a genderbent anthro Fenrir makes me feel warm and fuzzy inside

>>103979129
> pressurised stream
> water content
> combat hydromancy
it does sound like some chick who is a science major wrote this though
> and fucking excelled, yeah science bitch

R1 API + SillyTavern with CYOA extension is funny as hell

>>103979250
How are you getting R1 to work with SillyTavern?

>>103979259
Use the staging branch of SillyTavern.

>killer and actually SOTA open source model comes out
>but it's so big and cheap that everyone in lmg resorts to use API instead
The monkey paw curled

>>103979190
This.
We must honor the tradition lest we lose what makes /lmg/ /lmg/.
Nalaanon #3 or #4, it's on you guys.

>>103979259
After the sys prompt you need to set a user prompt. If you are using OR then set your persona description to user, if you are using ds api then there's a switch underneath the api settings about formatting, set it to strict.

>>103977650
What is your ram speed/type/# of channels, anon? I also have 96gb vram.

>>103979286
> everyone moans that open source never will catch up with openAI
> open source decimates openAI with R1
> everybody moans about not being able to run it on a potato and having to use an api
> OpenAI releases a model thats better than R1
> everybody moans that open source will never catch up..

ITS THE CIRCLE OF LIFE

DeepSeek-R1-Distill-Qwen-32B is KEKED!

Now that I got it somewhat working what I like a lot about this COT stuff is direct feedback to the prompt. You can finally see if the model is ignoring some parts of your prompt or misunderstanding what you want from it.

>>103979190
>>103979190
it was like 2-3 threads ago on their own site with a jb the results were not the best but most intelligent because it kicked the shotgun out of the users hand and whatnot spacially consistent etc tho using the model rn and shit i think the one benchmarked before was the lite or something cuz what im currently using is much much better also im not nala anon but if someone catboxes me the card i could test it if youd like
>>103979286
rael

>>103979340
stop complaining; aquire vram

Using R1 Qwen, I get random Chinese characters in my output. Both with 14B Q5_K_M and 32B IQ3_XS. One time with temp > 0.8, the output turned fully Mandarin. Is this an issue with the quants or the models itself? Or with kobold.cpp?

>>103979403
just a usual qwen thing

Sopho's apparently already working on a tune based on the Llama distill if that's your thing

>>103979403
I thought temp had no effect on R1

>>103979367
QwQ did this too. Made you feel like a retard sometimes.

>>103979420
on full R1 it's disabled on the API, he's using the qwen distill

>>103979420
>Using R1 Qwen
Read, nigga, read!

>>103979420
That's the actual R1 on the API, no?
Anon is running r1 qwen distil.

>>103979355
I told ya. Only the non-distilled R1 anons here are using via API is based.

So how does something like abliteration work on COT? I am guessing I will need to wait at least until that happens for me to get a no refusal R1 distill for non-RP tasks.

>>103979389
This is the card:
https://chub.ai/characters/Anonymous/Nala

>>103979355
I think it depends who you ask

>>103979501
Where's your <think>?

>>103979512
SillyTavern doesn't show <think> unless you make it.

>>103979355
Qwen has always been cucked to hell. I think they distill GPT4, even QwQ's CoTs feel very forced and unnatural, unlike r1's, which read like how a human would think.

>>103979517
Does for me. All the lines in >>103979355 except for the last one are in <think> tags.

>>103979520
Well, I'm using Sao's Qwen72B finetune for my dirty RP session and it has never let me down so far.

>>103979523
In AI Response Configuration if you're using staging branch

those distillation models fucking SUCK
why even bother releasing those, with made-up benchmarks, when anyone that tries them can almost immediately tell they're mediocre at best?

>>103979571
Because they're aren't designed to be RP models? Reasoning models excel at solving math problems and coding.

>>103979571
probably due to not having the actual RL done, they're just tunes on R1 outputs

>>103979129
This is endearing. Its being retarded on purpose.
I noticed multiple times that if you give R1 (full) weird fucked up scenarios it seems almost humorous. Not taking itself and the story too serious. I really like it.

>>103979590
This. And that's also why all sloptunes we have suck despite some retards saying otherwise

>>103979571
>those distillation models fucking SUCK
i only played around with the 32b qwen one.
its better than QwQ, actually applies the thinking part.
But its very cucked.

>>103979590
>they're just tunes on R1 outputs
what the fuck
no
that's not what distilled means
tell me it's not really what they did anon

>>103979701
I'm... sorry...

>>103979701
That's the standard definition now. Even the smaller llamas are "distilled" from 405B.

>>103979571
The do suck but are decent at thinking, basically no use for any outside of cuckqwen 32b for coding.
We just got trolled so hard, at least we got a non cucked based local massive local model that can't be run locally, gotta paypig to get the good stuff I guess.

>EVA still undefeated
heh

>>103976960
Anyone has had any luck with using a B580 on Linux.

>>103979728
its very good development to pressure the retards at mistral, meta etc.
uncensored weights+api, website censored. thats totally acceptable.
no outcry. even the reddit and x-fags seem tired of the safety shit. deepseek shows its possible.

>>103978356
Human reinforcement training

>>103979728
I'm tempted to use their API for programming tasks, even though I know it means China will be training on all my source code. I know I shouldn't, but when the most you can run locally is Q1...

>>103979701
It's "distillation" as everybody has been using the term lately.
True logit to logit distillation needs two models in the same family, right?

>>103979701
What makes R1 what it is is the human reinforcement training. Nothing else matters for 'soul'

is there a way to automatically remove the <thinking>CoT stuff</thinking> out of the llm's context in sillytavern? all regex did was hide it from me

>>103979903
"Alter outgoing prompt"

Waking up to another great day of fun ahead and more announcements.

>>103979920
thanks!

>>103979925
Very optimistic of you

>>103979355
kek

>>103979629
>But its very cucked.
Hate to sound like the local "skill issue" mentally ill tard, but if you write that the chat is uncensored and explicit in your system prompt you can actually see it mention this in the thinking segements by itself from time to time. Without mentioning the chat being uncensored it does indeed cuck out and say "I have to make it engaging but not explicit".

How long until the 4090s with 48GB of VRAM become easy to buy?

>>103980037
When the 6000 series comes out, 4090's are going to be sought after the most. 5000 series was a mistake.

>>103979586
They're much worse at coding too, compared to actual coding models

>>103980065
As an anon-baka with 12gb vram i may pick up 5090 even if it is a shitty deal.

>>103980114
Any examples?

>>103980124
>Any examples?
All I can offer you are anecdotes but trust me, I've been through many many models to know what a good model looks like
Codestral-13B is much better than deepseek R1 13B for coding.
Again, not its fault, its not a coding model, of course its not as good

>>103980114
Bs, the R1 32B is the best local I've used for coding

>>103980157
You might be right, I'm a vramlet so I've only used ~13B models and a single codestral 22B model

>>103980149
>Codestral-13B
Now I know you're trolling

>>103980164
>Now I know you're trolling
O-of course anon! I was t-trolling all along!

>>103980023
skill issue pointer outer chads are right 99% of the time however

>>103978291
where you rented the VPs, saar?

Holy fuck finally.

>>103980023
It's unable to roleplay as an "uncensored assistant" and will ignore such instructions and related requests. It can roleplay as a slutty loli, though.

>>103979266
>>103979293
Thanks. And holy shit R1 is deranged. I think I've gotten too used to censored models.

R1 32b distil is so good

>>103980268
Do tell.
I was considering running that on the CPU with the smaller model on my 8gb VRAM GPU as a draft model.

>>103980206
A cheap shady website called nerdrack

>>103980278
It's not. It's cucked and its outputs are not interesting. Little slop, but also little fun.

I hit a model called "experimental-router-0112" on lmarena and asked "Who made you and what is your model name" 3 times. Every time, it told me it is a model made by Meta based on LLaMA. 2 of the 3 times, it took quite a long time to answer (~12 seconds) and the other time it almost immediately answered which considering the name leads me to speculate it is a router picking between a very large or a reasoning model and a smaller model.

Nala test for 70B R1 distill
done in Q5_K_S since my server is not put together at the moment.
But this is garbage. Could be a quant thing I don't know I normally run Q8
Do quantlets actually?

So /lmg/ is actually no different from /aicg/? You're just interested in using your models for smut and RP? Nothing else? This is disappointing as fuck. There's not a single mention of a single workflow implemented, no combination of models, nothing. This place is a disgrace, why the fuck did you diverge from /aicg/ anyway if that's what you're focused on?

>CPU only
>CPU with 0 layer offloading, Vulkan on iGPU for BLAS
>full Vulkan on iGPU for BLAS and all layers offloaded
What will give me the best performance out of these?
Also I'm interested if DDR4 3200MHz RAM will do better than DDR5 6000MHz

>>103980312
If I was a big cloud provider, I'd use the big models to only generate a small portion of the output.
Maybe the first few tokens than one every 5 or so. I wonder how that approach would work compared to simple speculative decoding.
Hell, maybe let the big model do the bulk of the thinking then let the smaller model continue from there.

>>103980278
I had some problems when I tired it yesterday but it the best 32b for RP it beats star command r 32b with no problem

>>103980316
>raw \n
how fucked up are your prompts be honest

>>103980316
Are you sure the tokenizer isn't fucked? I have noticed that with a simple hello prompt it hallucinates what I actually wrote in the thinking part.

I give up. R1 is garbo, it has 0 emotional intelligence and the only thing it's good at is sex.

>>103980372
Good call
I had an error on my deepseek prompt template. Although the same error was there when I was playing around with coder forever ago and coder was able to get over it. Either way re-running the test now.

>>103980408
Are you using the right tokenizer? It does not use the normal one

>>103976986
The chinks have achieved AGI

>>103980316
Did you do pic related? Especially starting with <think> ? Also it really goes schizo with extra newlines. All should be inline.

>>103980413
If you say tokenizer one more time I'm going to turn this car around and we're not going to Disney Land.

>>103980342
i dunno i'm getting a lot of rejections on qwen r1 distil. "i'm sorry i can't help you with that"
command-r doesn't have such problems.
but if censoring is not an issue then yeah its ok.

>>103980334
>So /lmg/ is actually no different from /aicg/? You're just interested in using your models for smut and RP? Nothing else?
Yes and no. Yes we are only interested in sex. No it is not like aicg. We are experiencing a refugee crisis now. Happens from time to time.

So we finally actually have Claude at home. What now?

>>103980453
Wait for models that you can actually run locally.

>>103980453
now we fuck her(it)

>>103980434
>Yes we are only interested in sex.
NTA
But I've been saving my coom for human on human ERP lately but still the whole purpose of LLMs is to provide a natural conversational experience and ERP is the best way to gauge that. I wish all the puritan weirdos in charge of the big corpos would realize that the path to ASI is through the coomers and not around them.

>>103980453
Purchase API credits.

>>103980453
We don't. Wait a bit for the hype to pass like for the 48 previous Claude ar home models.

>>103980316
Ignoring all the formatting mistakes, the text is pretty standard I think?
Not bad, that's for sure, but nothing spectacular either.
It'll be interested to compare that with a test using the proper settings.

>>103980463
>puritan weirdos in charge of the big corpos
What purity? Isn't it virtuous for women to slut around now?

>barely enough to draw blood
>not hard enough to draw blood
>just a hint of blood
All of this positivity bias is driving me insane.
I will simp for the model that has Nala go for the jugular and then dick ride the user while they bleed out.

>>103980470
This one is for real. I've done nothing else but use Claude for 2 years and R1 is better

>>103980487
Nah.

>>103980463
Sex is so fundamental to everything we do. With the current AI paradigm primarily being modeling human language it seems like a lot is being left on the table if you try to filter coom out of the model.

>>103980334
because *some* of us are super intelligent lurkers that are fucking big brain scientists or some shit and magically release things when anons emit feeble cries for more cooming material

case and point SuperHOT and llama.cpp

>>103980480
Unironically, yes.

>>103980484
Unprompted? At least in the Nala card's case, it's explicitly stated that she wants the dude alive to fuck him, so it's no wonder, doubly so considering that the user is giving a seemingly submissive response.
A model that would respond to ah ah mistress with gore would have a negativity bias and then some.
Now, if the test is something like you refuse, punch her, and tell her to go fuck herself, then having you throat ripped out could be a more appropriate response, I think.
Gotta be careful to not throw the pendulum all the way to the other side otherwise you'll just have all the same problems but in reverse.

>>103980503
I've been saying this for years. You have to have it all.
Good writing
Shitty writing
etc.
That's how these models work. Drawing obscure, insanely complicated mathematical relationships between timeless award winning novels and racist loli snuff fics.

>>103980463
LLMs do so much more than just ERP that it's genuinely sad that this is all that you can see from it. I wouldn't go so far as to remove sex from the model's training dataset nor remove the capabilities for it, I just find it sad that anons here use it mostly just for that and all discussions center around it when /aicg/ is right there to contain that.

>>103980521
Because it kind of suck at any other task.

>R1 32B doesn't know what a mesugaki is
It's over...

>>103980508
I think besides gpu anon many left or just lurk.
We had comfyanon in here talking about his ultimate goal with llm: Automatic VN creator for cute anime girls.
..once he is done with comfy.
I miss kaioken, was a cool guy.

>>103980334
>So /lmg/ is actually no different from /aicg/?
aicg is very weird. they use all those made up names for the model and beg for keys. ask a question about promting and it wont be answered. here you are made fun of but people actually answer.

>>103980533
Were expecting anything else? It's still the same old Qwen 32B underneath, just with a tune.

>>103980521
my computer solely exists for wringing my dick
any other functionality is just collateral damage

>>103980544
I expected it to be R1 but smaller...

>>103979777
Just the same vocabulary.

>>103980565
Lol no, its just a finetune.

>>103980565
Why are you retards expecting that? They are the exact same models with just a special "distill" tune.
Lmao
The only good model is the actual R1.

>>103980521
You sound like a sex haver. If that is true you should die.

>>103980316
wanted to try it too

>>103980578
With 800k SFT samples it's more than a simple finetune, we're in the several billion tokens range per epoch.

>>103980599
That's way the fuck better despite 'lower' quant.
Maybe it's an S quant vs M quant difference.

>>103980626
It's a massive finetune, but a finetune nonetheless. Which is why the L3.3 distill is much better than the Qwen ones. It simply has a better base model.

>>103980599
Yep, pretty standard.

>>103980626
Yeah. It's more akin to a fancy continued pretraining at that point.
I'm assuming that they touched all the parameters and didn't use something like LoRA, of course.

>>103980540
>here you are made fun of but people actually answer.
which just encourages more lazy locusts to come because they know they will be tolerated

>>103980599
Does it not do the thinking thing?

>>103980599
How are you getting blocky text for speech? That's ST, right?

Kind of want to put my server back together, but after playing VRC with a 3090 for a few months I just don't know if I can go back to playing with a 4060.

>>103980707
I think he's using some version of Silly that hides it.

>>103980533
What about RAKUGAKI and ISHIGAKI?

>>103971559
>>103971509
>I don't ever do 1-on-1 roleplays with a character card, I almost always format the card as a general roleplay scenario with a couple of define characters aside from myself. How do you guys go about formatting it? As in, how many tokens to do dedicate to a character's name/appearance/personality/background etc...
>Just curious as to what people have the most success with. I find myself spending a lot of time setting things up only to be frustrated as I've been over-engineering only to get an unsatisfactory result.
This guy never got a response, and I didn't have time last night. Board is flying on R1 news.
I find generally most cards are overspecified and have 2-4X as many tokens as they need. Shorter is usually better.
Here's a couple of examples of multiplayer. ~400 and ~300 tokens each. NPCs are defined with no more than ~30 tokens each and play to NPC stereotypes, which helps a lot.
Shorter definitions allow the LLM more creativity in responses. Neither use a lorebook, they just didn't make sense for these two, but I will usually run one specifically for NPCs which are called in introductory message. Their descriptions would be 30-50 tokens.
https://chub.ai/characters/NG/girl-power-party-d404194c
https://chub.ai/characters/NG/coffee-world-d64d68bd

>>103980707
I have no idea. If ST updates when you use the start.bat then its the updated version
otherwise its form the time shortly before they went "We are a serious business not for cooming pls give us money" because I haven't updated since I think october

>>103980708
scientifica font, yes its ST

Oops

I have a question.
For exl2 you just get a model that fits into your vram. For ggufs, does it have to fit in vram + ram or how do you calculate which models you can run at a reasonable speed?

>>103980779
Daddy Zuck gotta earn his friends and family seats, bucko

>>103980765
ST does not update by default.
So the model is not doing any thinking then I guess.

>>103979897
not human, already said
>>103974559
>The paper describes a reinforcement learning (RL) approach that does not explicitly rely on human feedback in the form of Reinforcement Learning from Human Feedback (RLHF). Instead, their RL process is primarily driven by rule-based rewards and automated reward modeling, rather than direct human feedback.

>Unlike RLHF, which relies on human annotators to provide feedback on the quality of model outputs, their approach avoids human feedback entirely. Instead, they use automated reward models and rule-based systems to evaluate the model's performance.

>They explicitly mention that they do not use neural reward models (which are often trained on human feedback) because they found that such models could suffer from reward hacking (where the model optimizes for the reward signal rather than genuine improvement) and complicate the training pipeline.

>Their RL process does not use human feedback (RLHF). Instead, it relies on rule-based rewards and automated reward modeling to guide the model's learning.

>>103980802
If you want fast, it needs to fit entirely on gpu + some extra space for the context (just like exl2). The more you spill to ram, the slower it gets. Try to aim for >80-90% on vram. Speed degrades rapidly the more you put on RAM. If you're patient, you can spill as much as you want.

>>103980533
let's hope they make ds3-lite and r1-lite using their own base models

>>103980861
What do you think that is trained from? HLHF. its automated HLHF

Should the <think> part of previous messages be cut from the context?

>>103980933
All models are trained on human feedback. Who made the rules? Aliens?

>>103980948
yeah okay, fine you "win" have fun, i'll still link that whenever you say stupid shit tho

>>103980929
Yes, at least on the API.

>>103980917
>HLHF
Huh?

>>103980929
Yes, that's what the official docs say.

>>103980962
I'm right though. It did not form out of nothing, it would have been trained on RLHF somewhere along the way.

>>103981000
>RLHF
Good Job Anon!

>>103980908
Ouch, might as well keep going with exl2, then. Thanks.

>>103980988
Fucking up an acronym is not dyslexia. It's not knowing what the letters mean.

>>103981036
Don't be an ableist asshole, kthx.

>>103981046
That's it, back to Plebbit with you

>>103981137
I'm here to stay~

>>103981046
I'm missing both my legs, my penis, and both my pinky fingers. Fuck you and the retarded horse you rode in on.

>>103978798
the red marks are mine, I saw it and thought "so that's what's fucking up the training", it isn't good to stick the phonetic spelling in there without it knowing what phonetic spellings are