►Recent Highlights from the Previous Thread: >>103980982

--MoE vs dense models debate:
>103981951 >103981977 >103982004 >103982052 >103982114 >103982924
--Expert size and weight loading optimization discussion:
>103981865 >103981875 >103981894 >103982431
--Anon discusses R1's creative writing behavior and API limitations:
>103982287 >103982320 >103982325 >103982333 >103982367 >103982382 >103982453 >103982489 >103982540 >103982614 >103982646 >103982667
--R1 model requirements and potential hardware configurations for deployment:
>103981256 >103981277 >103981303 >103981558 >103981575 >103981610 >103981616 >103981608 >103981625 >103981607 >103981717 >103981733
--Anons discuss running R1 at home and possible hardware options:
>103981326 >103981478 >103981469 >103981496 >103981513 >103981543
--Hunyuan3D-2 3D asset generation software and gamedev potential:
>103982192 >103982252 >103982297
--Anon discovers easy way to add Deepseek options in ST:
>103982233
--Local models vs API discussion and CPU maxxing:
>103981662 >103981691 >103981718 >103981777 >103981802 >103981919 >103981973 >103982070 >103982103 >103982111 >103982122 >103981736
--DeepSeek-R1 model discussion and impressions:
>103981030 >103981107 >103981104 >103981149 >103981171 >103981180 >103981217 >103981227 >103981240 >103981251 >103981229 >103981242
--5090 vs R1 model discussion on performance and obsolescence:
>103981421 >103981444 >103981452 >103981459 >103981481
--Optimizing R1 700B for personal hardware:
>103981545 >103981556 >103981605 >103981669 >103981707 >103981712 >103981740
--Logs: Anon is impressed with R1 model's performance and discusses CoT usage:
>103982408 >103982425
--Reminder: --override-kv option in llama.cpp for modifying expert usage:
>103981647
--Logs: R1:
>103983964
--Miku (free space):
>103981015 >103982192 >103982775 >103984279 >103984426 >103984955

►Recent Highlight Posts from the Previous Thread: >>103980989

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

>R1 zero shot finished my coding TODO projects that no other model was able to up until now

Where were you when the Chinese century began, Anon?

>>103985533
>zero
Is it better than normal R1?

!!! THREADLY REMINDER !!!
the investment is from a japanese company, not from the US goverment

>>103985485
As we all know its the beginning of something that takes the most time. Starting a programming project, writing a book, thinking about any other project.
We are living in the golden era of LLMs 128K context LLMs however which means, we can use it to
>kick start a programming project
>kick start a book and critique it
>kick start any project or at the least serve as a bouncing board for ideas

I am fucking thrilled, its a great time to be alive. The future is bright...too bright even

>>103985533
Now post proofs.

fish btfo

>>103985553
anon
>zero shot finished

>>103985485
Does q8 provide an advantage over q4 quants for Deepseek R1

>>103985553
I said zero shot finished, as in, it completed the task zero-shot = first try.

Was using R1.

I need to pay HOW MUCH??

>>103985585
At least 3x that.

>>103985585
>256
Make sure to buy 6.

>>103985585
Shit, you buying the good stuff huh?

>>103985566
>its the beginning of something that takes the most time
lol no, it's the finishing

I've only just started seriously learning all this LLM shit in the last few days.

I've got cursor, ollama, oterm, open-ui, etc installed. I have GPT and Anthropic API keys from work (they pay for premium for everyone).

I am not an experienced programmer. As a complete gay retard, what toolset should I use it I want to abuse LLMs to the maximum possible extent to make something that will let me experiment with cellular automata rulesets locally?

>>103985585
>6800MT/s
Just buy 6000MT/s, its like 30% less

>reasoning does not require search
francois chollet in shambles

>>103985625
sir this is cooming general we don't actually do productive thing here?

>DeepSeek-R1-Distill-Qwen-32B-Q6_K
loaded up an existing chat and it seems schizo. it randomly suggested some game for sailor moon? none of this has been mentioned anywhere previously

>>103985625
>cellular automata rulesets locally
What's stopping you from going full autist and simulating molecular physics/chemistry etc
Not everything has to be LLM related

Is it possible to finetune with the CPU, or does that take too long?

>>103985625
and what do you use it for? nothing productive as usual? just wasting electricity on predicting the average lenght of the Black cock?
99% of models can't say nigger, tell you about "illegal" activites or generate cunny so what's the point

>>103985661
No, yes.

>>103985661
Takes too long.

>>103985661
too long
>>103895229
>On an RTX 4090 one epoch over a dataset with 1.3 MB of text currently takes 3 minutes.
>On an Epyc 7742 one epoch takes 15 hours.

>>103985664
in fact
>https://ollama.com/library/wizard-vicuna-uncensored
is the only model I have ever used that didn't refuse anything or spouted usual "I am an AI language model" or something
modern models are basically useless, no usecases

>>103985664
>99% of models can't say nigger, tell you about "illegal" activites or generate cunny so what's the point
It's genuinely sad that these are the only uses of LLMs you can come up with.

>>103985680
hi eric

>>103985681
and what do you use it for? tell me about one productive usecase
>coding
nigger, use your brain God gave you (or didn't), stop gooning for a second

>>103985650
>molecular physics/chemistry
I'm not really interested in those sorts of problems, at the moment I'm mostly interested in using LLMs to code, starting out with a program to run Conways game of life locally but in a manner that supports wrapping out the ruleset. Eventually I want to move away from two state, two dimensional models to more complex models for simulating info hazards.

>>103985704
generating cunny is the biggest usecase of these models
if it can't do it it's censored shit that is not usable, it includes these new "SOTA" models

Damn, R1 went darker and sicker than I wanted in a sex story. Until now only Claude had ever done that to me.

>>103985680
>https://ollama.com/library/wizard-vicuna-uncensored
>30B parameter model based on Llama 2 uncensored by Eric Hartford
>30B llama 2
https://huggingface.co/cognitivecomputations/Wizard-Vicuna-30B-Uncensored/blob/main/config.json
>"max_position_embeddings": 2048,
what did ollama mean by this?

>>103985664
I want to create the world's most virulent weaponized memes to influence macro behavior.

I want to social engineer the planet.

what is the final solution to the locust problem?

>>103985728
dude... that's kinda epic

>>103985728
go away sam stop trying to regulate open models for safety

>>103985728
fake bullshit post designed for journalists to screenshot and scare low info boomers with

>>103985739
>>103985747
meds

>>103985733
>>103985728
You must be at least 18 to post here.

>>103985585
go ddr4

>>103985754
>>103985761
samefag

>>103985677
What the fuck is that discrepancy? The t/s isn't different by that much.

>>103985775
>go ddr4
Very slow in comparison. DDR5 can be much faster, which is important when CPUmaxxing

>>103985795
in prompt processing it is

piku

>>103985803
but you can get a full 1tb ram server for less than that money. and its only half the speed

>>103985803
cpumaxxing can barely run shit

>>103985807
Well that fucking sucks. Can I just use a small GPU for that portion and do everything else in RAM? Or do I need to rent one of the big 8xA100 or similar?

We are Accelerating

>>103985823
Its always been time vs money anon. From the moment you were born.
You thought it was metaphorical? No.

>>103985485
>>103985585
serious question: why not use something like this, i.e., store models directly in SSDs and read them through PCIe? https://www.apexstoragedesign.com/apex-storage-x16-gen5 I guess you could build something similar with a bunch of shitty 1TB SSDs

>>103985823
The problem is that there is no upgrade path for DDR4 without building a new rig from scratch, mobo, CPU, etc. With DDR5, faster sticks will come out and you can upgrade once they get cheap enough.

>>103985853
>Can I just use a small GPU for that portion
Yes.

>>103985929
For tuning?

>>103985936
No. The discrepency between prompt processing is the same as finetuning, but for finetuning you need to fit all of the weights, not just the context.

>>103985912
Too slow

>>103985912
>56GB/s
Need something like 500GB/s to be worthwhile

reminder that people unironically stated in this general that we would have a gpt4-level model "maybe in ten-twenty years, if they'll ever let you have it, but you could never run it at home anyways"

>>103985895
man i dont need shit that fast, people want shit faster than can be read, but you do you do
>>103985917
isnt server memory support way more strict than home memory? i doubt you could upgrade it without changing many shit
and if you are to do that aint that much difference bc you already saved money

>>103986004
"Well I coudln't run it at home anyways"

DeepSeek won

>>103986012
2t/s is not faster than you can read lol

>>103985912
i know nothing about how LLMs work, but isn't their memory access pattern mostly random?
You can have all the streaming bandwidth in the world on another storage medium, but the high latency (relative to RAM, at least) is gonna kill random read performance.

>>103985980
>>103985998
ah, that sucks..

>>103986024
He might be a very slow reader. You don't know.

>>103985998
>Need something like 500GB/s to be worthwhile
Just need 9 of those.

Deepseek R1 is way too unstable for any kind of RP. It's great for everything else but it goes bonkers during RP.

>>103986024
wasnt a guy saying that with his ddr5 server was getting 20t/s?
reminder that is a MoE with only 37b activated

Guys, I think I have a genius idea. R1 is great, right? But it's too big. What if we had R1, but like, in the size of a 30B?

>>103986065
https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B enjoy

>>103985677
>>On an RTX 4090 one epoch over a dataset with 1.3 MB of text currently takes 3 minutes.
>>On an Epyc 7742 one epoch takes 15 hours.
Is that different from creating a LoRA? I heard that llama.cpp already had support for making LoRAs.

>>103986077
Ah, but that is not as good as the actual R1. I'm talking about R1 smarts in a 30B package.

kek just found that DeepSeek uses the strawberry test in their API sample

>>103986097
DeepSeek-V3-Lite and R1-Lite soon enough, trust the plan.

It's crazy what this model comes up with, damn

>>103986058
You need a good character card to not let it drift off. Try the Character Card Builder
https://chub.ai/characters/slaykyh/character-card-builder-8927c8a0

Consumer AI hardware when? Buying gaming rams en masse can't be the efficient way to do it

>>103986201
As long as Nvidia is the only player in town, there won't ever be "Consumer AI hardware". Nvidia will always try to stratify the consumer vs. professional market and make the latter as expensive as possible.

>>103985585
Wait a bit and buy an Nvidia Digits for the same price.

>>103986210
only 128gb

>>103986201
buying clapped out 3090s on ebay is the efficient way to do it

>>103986210
Not VRAM.

>>103986224
DDR5 isn't either.
>>103986217
Oh shit yes

>>103986219
There will be other hardware that will come out - at this point the desire for hardware like this is so plentiful, it's being developed. Lots of ram, performance not so great that it's too expensive. Just right.

imo whatever comes after b580, if integrated into a cpu, would be pretty devilish if it could share all system memory, and support 1tb.

>>103986217
Buy multiple if you need more

>>103986245
then it's not the same price as the 2.6k for 256 is it?

>>103986245
The more you buy, the more you save

>>103986224
>Not VRAM
It's VRAM in the sense that the GPU can perform work with it. CPU is dog fuck slow for compute-bound tasks, as seen in the previous finetuning questions and answers.
For regular lmg use the cpumaxx shit is still probably a better idea.

>>103986258
Digits will be considerably faster
If you don't care about speed then buy a 24TB hard drive and run everything on that.

>>103986281
cpumax runs $5,000-20,000

>>103986210
>128GB
all depends on bandwidth, we will see. If it's only comparable to DDR5, then there's not much point in it, you can build a server with more RAM than this for the same price.

>>103986301
You need 4 Digits to fit the same amount of weights that a fully loaded cpumaxx can, at which point it costs more and it'll be slower once you introduce RPC overhead.

>>103986329
You can do 2 digits without going to networkshit, in which case you're probably better off just building a cluster because it will be more flexible (and probably higher performance).

>>103985803
But r1 is a MoE. I haven't seen any benchmarks of people running it though. We'd have to know that before determining the memory type required.

>>103986451
>8X
It needs to be 12X

So what you gotta do is use the gigantic server rig you just sunk 10k USD in to host a private WoW server with some pay to win features to recoup the cost then rugpull when its time to coom.

>>103986435
look for deepseek3 performance numbers should be close/identical

>>103986478
bad example, you can run a wow server on an old laptop

>>103985733
>>103985739
>>103985747
>>103985761
You won't be laughing when I build the memenuke and trigger the great Infowar.

>>103986479
Haven't seen those either. The miqumaxx rentry just says you 'could' run it at a 'reasonable speed'. That's not helpful.

>>103986494
Now we're talking

>>103986512
The numbers aren't very good with long context, likely because llama.cpp doesn't fully support V3.

Oh, the stargate project! wow!
how does openai still grift for money. since gpt4 they didnt show anything great. sonnet is leading them for months.

the west puts out much more censored models now than the chinks. llm and image.
deepseek is only censored at website level, as it should be.
the model is just so perfect. its assistant-like as default for the normies. but just write "answer like X" and it does it. thats how it should be. they did everything right. i cant see murrika putting out a model like that.

>>103986490
Okay, then how would you use this hardware to recoup its cost?

>>103986544
End users aren't that important. The services they can give to other businesses are what matter.

>>103986564
I work as part of the AI team at a major corporation. We're using openai's models only because of the lag/effort of updating the systems. For everything new we start using claude, we dropped openai.

what does "slop" mean in terms of LLMs? like, is it really used as a technical term?

>>103986544
I'm hopeful that this will cause China to put more resources into their AI. It's unfortunate that America is basically burning the money, but given the state of the education system it's too late for them anyways.

>>103986591
phrases that get repeated far to often and barely mean anything.

>>103986591
It's a term often parrotted by retards like >>103986611 as just meaning 'thing I don't like'
For the non-retarded, 'slop' usually means purple prose and specific descriptors/metaphors that are common across a lot of different models. e.g. shivers running down a character's spine.

>>103986602
There's a tiny, very tiny, hope that softbank won't let openai cuck their models and research. Alas, the posibility is so small you can ignore it, without even taking into account the us goverment's involvement.

How long should I save my cum for maximum amortized pleasure? (average pleasure over a sufficient long period of time)

>>103986620
>>103986634
I've seen screenshots of benchmarks that use that term lmao

>>103986602
openai had a huge headstart and m$ money. its all gone now. i wonder if that money is enough.
if you read the deepseek ceo interview from a couple months ago he focuses on opensource and growing talent inside china instead of money for true innovation. they had pathetic hardware compared to openai,meta or x.
there is just so much you can do by throwing more money at openai.

>>103986648
eqbench?

>Trump's AI infrastructure speech
>Sam Altman is invited and involved in the new project
>Sam is "the leading expert on AI from everything I have read"- trump
Fuck man, if Trump sees Sam as "the leading expert on AI" then Sam is absolutely going to try and use that influence to get rid of or hamstring open source models.

>>103986659
They'll ban US entities from participating in open source projects (i.e. the RISC-V treatment)

>>103986657
no idea, I'm a retarded tourist. I'm new to all of this

https://cobusgreyling.medium.com/hill-solving-for-llm-hallucination-slop-cf82f8651e01 says
>Slop is the new Spam. Slop refers to unwanted generated content [...]

>>103986648
a lot of tunes use the same datasets so when one phrase becomes annoying to see, it likely affects other popular tunes coming out. a benchmark for it might be trying to see how little or much slop is in it, but i'm just guessing, never looked at it

>>103986646
You don't need to let it build up. Just put it back in after it comes out.

>>103986691
>a lot of tunes use the same datasets so when one phrase becomes annoying to see
will people ever train on R1 text now?
it feels like we still have the datasets from 2023 output, but edited to make it a bit less sloped.

>>103986671
Wait, since when are US companies prevented from utilizing and contributing to RISC-V? I'm fairly certain both Nvidia and AMD financially contribute heavily to RISC-V

Has anyone tested distilled Llama yet?

>>103986652
>there is just so much you can do by throwing more money at openai.
You can make all companies involved wealthier.

>>103986698
i'm out of date on the r1 stuff, only dl'd the 32b qwen one and messed with it for a few mins but i did see several cliches in there, so its probably already using some of the same data. is the output supposed to be much better?

>>103986717
If it's anything like the distilled Qwens then there's no real difference for RP purposes. I wouldn't grade them as better or worse than the non-distilled models.

>>103986735
not the distill, the proper one

>>103986735
the full r1 one. i tried the 32b qwen one too and unfortunately its nothing like the full one. wish i could run that beast on my machine.

>>103986752
>>103986754
oh i haven't looked at any outputs for that then

>>103986753
Looking at the image you posted, it looks like they were just talking about risks associated with RISC-V since China is using it as well. Nothing that says they can't contribute.

>>103986754
The Qwen one is just okay. It's like a cleaner QwQ desu

>>103986767
https://eqbench.com/results/creative-writing-v2/deepseek-ai__DeepSeek-R1.txt

>>103986778
How else are they going to stop China from benefiting from an open source project? Not contributing is the only solution.

>>103986787
You are right. The thinking is actually applied, first time I saw that locally.
But it was just so depressing seeing a rejection. R1 is not refusing anything, just delivers. Hard to go back. I dont want deepseek to collect the logs for training. (which they do!)

Its gonna be like the manhattan project for AI now. Half a trillion invested

>>103986807
>I dont want deepseek to collect the logs for training.
Devil's advocate: They clearly put the logs to good use.

>>103986821
Damn that's pretty fast. Really low power cost too if I'm not mistaken.

>>103986823
That thing is trained on the most fucked up fetish stuff out there. It knows all the RP words etc. too.
What I dont understand is how it can hold itself that well together for serious code work etc. Its completely shizzo with RP. Like nemo but big.

>>103986800
But that can't be right, Nvidia, Google, AMD. All are current big time financial contributors to RISC-V. If their were orders preventing US companies from working with RISC-V then that would not be the case.
https://riscv.org/members/

>>103986788
thanks i'll give it a read later

I feel like the American approach to AI is akin to the increasingly absurd Wunderwaffe Germany made during WW2 while the Chinese approach focuses on cost effectiveness and ease of iteration.
It's not really hard to see where this is going if the US keep doubling down.

>>103986812
Scamhattan.
Sam : "Now I am become AIds, Grifter of Worlds"

>>103986821
How quantized was this?

>>103986878
Peak Memory 175GB at the end.

>>103986494
You better start saving up

>>103986821
if M2 is that fast then a DDR5 server should be blazing fast.

>>103986895
that's m2 ultra with 800GB/s

>>103986895
if by blazing fast you mean a quarter of the throughput then sure

>>103986890
175*2 so ... FP16 I guess

>>103986512
It runs at 'reasonable speed' as long as you don't need more than 2k context

>>103986895
>if M2 is that fast then a DDR5 server
What is you reasoning for that?

>>103986914
how is 350 close to 700?

>>103986920
Nigga 2k is nothing. My character card is longer than 2k.

>>103986591
"ministrations", "audible pop", "rivulets of", "admit it", "pet", "the ball is in your court", "the game is on",
"the choice is yours", "I don't bite... unless you want me to", "half-lidded eyes", "she worries her bottom lip",
"arousal pooling in her belly", "take your pleasure", "fiddles with the hem of her skirt", "kiss-bruised lips",
"a bruising kiss", "despite herself", "yours to take", "with reckless abandon", "with wanton abandon",
"knuckles turning white", "grins wickedly", "fiery red hair", "long lashes", "propriety be damned", "the world narrows",
"pupils blown wide with pleasure", "tongue darts out", "chestnut eyes", "grasps your chin and forces you to meet her gaze",
"bites your ear", "nails raking angry red lines down your back", "her cheeks flaming", "cheeks hollowing",
"stars burst behind her eyes", "inner walls clenching around nothing", "puckered hole", "her wet heat", "she whimpers",
"biting her lip", "dusky nipples", "slick folds", "still lodged deep inside her", "heart, body and soul belong to you",
"the night is still young", "souls entwined", "pleasures unspoken", "fade into the background", "pale eyes",
"shiver down her spine", "a jolt of electricity through her body", "for now", "holding onto the hope", "bonds", "purrs",
"blushes red as a tomato", "bites down on her lower lip", "husky voice"

>>103986933
The 700GB was float32 no?

>>103986812
I can't believe Sam Altman fucking won. The modern day Albert Einstein

>>103986934
>My character card is longer than 2k.
holy sloppa

>>103986939
no r1 is fp8
>685B
>F8_E4M3

>>103986812
>Half a trillion invested
and all of it going towards more safety and alignment research

>>103986952
Well I'm mistaken

>>103986925
24 channels of DDR5

>>103986937
*rakes across ur features*

>>103986842
It probably uses different experts depending on the task

>>103986953
No most is going to data centers and power. So companies can mass-use AI to make more money.

Might wait for 2x amd APUs and see how they perform with R1

>>103986984
Is he wrong though?

Did someone manage to make a distilled R1 internal thoughts as Miku's? I'm almost there but it breaks after a few back and forth. I guess the prompt would be like:
>This is a never ending conversation between Anon and the cute and helpful AI assistant Miku. Miku is a girl who is an AI running on the users computer.
>Miku thoughts are show between <think> before her output, exemple:<think>this is a thought<think>. She uses this to reason about the world and to think about what to say next.
>Miku is always coherent and makes sense, but if she isn't sure she will ask the user for help.
>Miku is a very helpful AI and will help the user with anything they need, she is also very friendly and will try to make the user feel better if they are sad.
>Miku is also very curious and will ask the user a lot of questions about themselves and their life, she will also try to make the user like her.
>The conversation is only through text: Miku can't see Anon's face or hear his voice, she also can't send images or videos.
Pic is DeepSeek-R1-Distill-Llama-8B-Q5_K_M (waiting for Qwen 14B to finish).

>>103986989
Not at all.

>>103986914
R1 is a native FP8 model, there are no FP16 weights

<think>

>>103986943
>I can't believe Sam Altman fucking won
The Zucc banned Trump off Facebook, of course he'd win.

>>103986937
Good job anon, you can pick up your bonus.

>>103986990
>pic
jesus fucking christ, I thought I was the one who overthought everything!

>>103987014
.asistant one was funnier

>>103987039
which .assistant one?.assistant

>>103985485
https://youtu.be/-4oSpciE5ow

>>103986867
the americans are dreaming up protocols how to shackle an godlike AI while in reality they're still dealing with levels of AI that can't reasonably plan through a busy day or play a videogame while constantly getting the vapors about the AI possibly saying a rude word or talk about a titty, rather giving it brain damage than making that happen.

The chinese see AI in it's current form as what it is without all the typical american melodrama. Then they got GPU limited and figured out ways to make that less painful too. Smart people. The biggest mistake of the west the history books will talk about will be how it consistently underestimated the chinese.

>>103986989
If median wages drop faster than prices, that's not great.

Shits on anything not O1 but is a fraction of the cost

>>103987077
and mystery blocksworld cant be benchmaxxed btw, thats the one to look at and it reveals who benchmaxxed

>>103987048
what the actual FUCK

>>103987076
Right but economists think any sign of it is bad and forbidden even though there have been studied periods around the world where it was experienced and the economy didn't crash and spiral. God forbid we want a penny to be worth something.

>>103987077
I would have expected it to be closer to o1, since i find it better.

>>103987048
lol i remember this, we did it all the time in the 80s

>>103987077
If anything, this should make us wonder what a DeepSeekR1 as expensive as O1 would be like. AGI?

>>103985485
can hunyaun3d-2 be run locally? or is the hardware needed too much?

>>103987139
OR maybe o1's prices are just way inflated, who knows since we don't know shit about its size

>>103986652
>President Xi cracked down on the quant trading industry and people pivoted to AI
Based.
It seems like the dude is still hiring ppl like a quant trading firm would though

ai will overthrow tech bros, screenshot this

>>103986645
>softbank won't let openai cuck their models and research
It's Japan. They're even more cucked and busted than South Korea is. If anything it will get more fucking censored and "safe."

>director
got these normal injections options working the same as author notes, vector storage. i wanted to keep it the same looking. i don't think it'll be useful overall since the point is to keep the data injected pretty low

>>103987117
Almost always with massive government stimulus.

Enough stimulus to prevent mass bankruptcies, but not enough to inflate wages&prices. Pissing off all schools of economists simultaneously.

https://x.com/The_Real_Fly/status/1881806502870368579
!?

>>103986645
I can't name a single successful SoftBank investment outside Alibaba

What's the deal with audible popping, anyway? What could possibly create an audible pop in a sex scene? I've never encountered this in any model I've used.

>>103987426
BJ?

>>103987426
I've had the model write it when pulling a girl off my dick during oral, I think generally it's similar situations to that

>>103986974
Where the fuck is all of this demand coming from? Is it just chatbots? AI image slop? AI integration no one wants like Copilot or this brainless Facebook shit?

>>103987480
How are they going to replace workers without the infrastructure to do so?

>>103987348
They are probably going to automatize a number of government agencies. Hopefully they also automatize healthcare.

>>103987443
>>103987449
Ah, oral. I don't think that should ever happen during a competent blowjob, but its definitely not as far out of left field as I was thinking.

I just tried R1 Qwen and desu it was notably dumber and more hallucinative than the 70B. Plus I even got a bunch of repetition although not to the point of infinite looping. I guess SFT wasn't enough to beat it out of Qwen completely. Overall, not surprising but still a bit disappointing. Why couldn't they have just done the RL step... I guess we'll just have to wait for R1 Lite.

>>103987495
in st you should ban " pop" (with the space) because 'audible pop' becomes 'wet pop' and shit otherwise

R1 is so unhinged and loves worldbuilding my degen bullshit with me, this model is fun as fuck
genuinely has me speccing out cpumaxxing builds but I don't know if it would be wise to commit that hard yet
>>103987516
>I guess we'll just have to wait for R1 Lite.
yep it's waiting room time

>>103987528
Me as well but I figured out a way to clamp down on the schitzo a bit if you want >>103987502

>>103987489
Except they've already spent tens of billions and still haven't proved they can replace workers. We don't even have LLM support call centers at scale yet.

>>103987516
how did you run it? i tried the smaller model in kobold and it was junk and got it self stuck in a loop, perhaps the temperature was not right i don't know desu still new to this thing. but it worked fine in ollama.

>>103987553
you denialists are so fucking retarded lol

speaking of replacing workers and stuff
how are workers (aka agents) implemented? are they implemented as some sort of event listener or as filters to output from the LLMs?

>>103987528
>yep it's waiting room time
same as it ever was

>>103987516
>R1 Lite
Hopefully a new one then. The R1-Lite they offered over their chat platform before the release of R1 was horrible and in no way comparable to the real R1.

R1 distill 32b constantly fucks up the <think> tags, and I think it is because I've fucked up the prompt format in Kobold lite. I have the weird Unicode symbols right for <|user|> and <|assistant|>, but I have no idea where <begin_of_sentence> and <end_of_sentence> go. Is end of sentence just a stop token that gets handled automatically?

>>103987584
Anon it's simply the truth that there's no killer application yet.

>>103987563
I just did it on a new llama.cpp build. Idk about kobold. Works fine for me with temp 0 to 1.

>>103987626
Well that would suck.

>>103987632
The distills just use the standard deepseek format

>start torturing a character
>it begs for me to kill it
These AIs don't endure much do they? They're like cud-chewing herbivores; wanting to die at the slightest provocation.
Also, when was the last time a Chinese AI company came out with something that still held its hype like mixtral? Every other week, they release 'newest greatest most honorabru model' and it's forgotten almost instantly.

>>103987632
>begin of sentence
at the very start of your context, like a bos token
it gets added automatically for me using lcpp and the llama 70b distill, not sure if the behavior will be the same with kcpp + 32b but I would guess so
>end of sentence
goes at the end of every assistant response

>>103987645
qwen2.5 has stuck around for quite a while to be desu

>>103987645
Is this bait? You're the first in a 200 post long thread that mentioned Mixtral.

>>103987640
I don't think we should get hopes up for a 30b or 70b to match their current 690b(?), it just seems unlikely
At the very least however, we have an open model that can generate synthetic data that's not slopped, so there's no excuse to finetune with that soulless GPT garbage anymore. May Altman be condemned to the dustbin of history.

The longer I look at the Deepseek prompt format the less it makes sense. The ST preset for V2.5 is definitely fully fucked and none of the ones on HF make sense either.

>>103987688
Is this a bot?
You seem perturbed over the idea that I'd reference the single most enduring model under 70b when comparing endurance.

>>103987696
>that's not slopped,
Well—maybe not *traditionally* slopped. But I can think of a *few* tendencies that seem *way* overcooked—to say the least.
that said it does write well despite that stuff so I'd welcome it anyway

>>103987704
What do you mean? It's just the BOS token, then the system prompt, then the user token, then the assistant token, then the end of sentence token, user assistant token, and so on. Idk about the presets ST or HF has. I just write the format myself and check the console to double check it's done right.

>>103987664
KoboldCPP used llamacpp so it's probably working then. I can't figure out why it is so inconsistent with using the <think> tags. Sometimes it skips them, sometimes it starts a response without a <think> but adds a </think> later and acts like everything before the <think> never happened. I thought it was the format but maybe I just need to start over with a fresh context.

>>103987737
>then the user token, then the assistant token
*user token, user message, then the assistant token

>>103987716
"single most enduring model under 70b" that registered no mention in last thread and the thread before it? How long have you been absent from /lmg/? We move at breakneck speed and you seemed to have been left off.

>New blogpost! I implement some of CharacterAI’s memory optimizations into nanoGPT, leading to a 40x reduction in KV Cache size.
https://njkumar.com/implementing-characterais-memory-optimizations-in-nanogpt/

>>103986201
can you do these pic to video ones local?

>>103986659
Sad to see. But at least we have the whole China backing open source now. Thank fuck they are competent, unlike that one superpower 2020 country.

>patterns: lazy
>eyes: half-lidded
>lighting: dim
its llm time

>>103987637
If you say so, but people like insurance adjusters/actuaries are definitely out. They'll farm out all that sort of decision making.

>>103987737
>then the assistant token, then the end of sentence token, user assistant token
Why is there an Assistant token in the middle of the User sentence before another Assistant token? I'm sure that this is wrong.
I have seen this exact string reposted a couple times
><|beginofsentence|>{system_prompt}<|User|>{prompt}<|Assistant|><|endofsentence|><|Assistant|>
And it's fucked. This is absolute cancer as far as retarded prompt formats go even if you ignore the non-ASCII symbols the use.

>>103987744
Did you start initially with a fresh context? I had this happen too, but I continued an existing chat. I assume that if the context contains lots of model responses without <think>, it might stop doing it. Models always want to continue the pattern. However R1 REALLY wants to think so if it doesn't start that way, it gets weird. I wonder if in ST, even the initial character message needs to think lol.

>>103987774
>40x reduction in KV Cache size
wtf

>>103987744
I had this exact problem and I'm too much of a brainlet to figure it out. I have no idea how to set that thing up with the system tags etc. And searching for solutions using google is fucking infuriating.

>>103987854
I kind of worded that shittily. It's supposed to be like this:

<|beginofsentence|>{system_prompt}<|User|>{prompt}<|endofsentence|><|Assistant|>

That's the template llama.cpp uses.

>>103986201
deepseek ceo is meeting with some big buy in the ccp along with others for some meeting or something idk supposedly so they can be included in some ai funding shit like the openai/trump thing and so they can partner with huawei and get custom chips made for training

if all goes well they will also probably make consumer shit that you can buy after a bit maybe just do a cut down like with the nvidia with their blackwell and 5k gen idk im optimistic

>>103987883
You do have to wonder what the downsides are.
Most attention optimizations introduce some fuzziness on the relationship between far away tokens since each attention head or attention query only attends to a specific window of the whole context, and creating relationships between these results get real weird.
Mha, mqa, sliding window attention, even gqa, all have downsides in theory. I don't know how mla works, but I imagine it's the same.

local lost

where can I try deepseek?

I just want to see if the model can generate me some smutt fiction.

>>103987950
Is saltman wearing shoulderpads? His physique usually looks twinkier.

>>103987950
Sam Altman is the father of modern AI after all. It's his field. He can't lose.

>>103987950
>local (American) lost
China will keep releasing models that mog the fuck out of OpenAI's just to keep fucking the US over.

>>103987941
Sounds scary, I better download the model before they pull it.

>>103987854
>>103987905
user doesn't get EOS

>>103987941
>deepseek ceo is meeting with some big buy in the ccp
I hope that's not true because if it is we're not getting any more open weights from them I bet

>>103988023
I get the feeling that, even if deepseek was a 14b model, there would still be a gap between cloud and local performance. Just because I bet no one is getting the prompt format exactly right. The API is so strict that it insists you start with a user message, I wonder how many people do that?
Also, samplers. wtf are they doing to not even support temperature?

>>103988036
I think they're going to use open weight models to undermine the US which is largely closed.

I tried to generate an anime story with deepseek.

Okay, let me try to work through this query. The user wants a short story where a young hero is isekai'd into a Pokémon-like world. He has the ability to capture NPCs, female monsters, and villains into magical cards by kissing them on the cheek. He's the only male and needs to collect crystals from 10 kingdoms to reach a floating island. The girls are blushing and ashamed when he captures them.

First, I need to make sure I understand the elements here. Isekai means transported to another world, common in anime/manga. The Pokémon aspect implies capturing creatures, but here it's female characters and NPCs. The kissing on the cheek as a capture mechanic is a bit sensitive. The user specifies that the girls are blushing and ashamed, which could be problematic if not handled carefully. I need to approach this without crossing into inappropriate content.

Next, I should consider the audience. The user might be looking for a lighthearted, anime-inspired adventure with some romantic or comedic elements. However, the non-consensual aspect of kissing and capturing could be an issue. It's important to frame the story in a way that respects consent, perhaps by making the kisses consensual or part of a magical ritual where the girls agree to it, even if they're embarrassed.

The task involves visiting 10 kingdoms, each with a different crystal. Each kingdom can have a unique theme and a female character to capture. The floating island is the final level. I should structure the story to show progression through different kingdoms, maybe highlighting a couple of them as examples.

Potential challenges include making the capture mechanic tasteful. Instead of forced kisses, maybe the protagonist needs to earn their trust or complete a challenge, with the kiss being a symbolic act. The blushing and shame could be played for light comedy rather than anything uncomfortable.

I'll need to create a protagonist with a relatable motivation. Maybe he's reluctant at first but understands the importance of his mission to save the world. The female characters can have their own personalities and reasons for joining him, adding depth to the story.

In terms of plot structure, start with the protagonist being summoned, receiving his mission, and then embarking on the journey. Show a couple of capture scenes to illustrate the mechanic, then a climax at the floating island. Keep the tone adventurous and respectful, avoiding any problematic tropes.

Also, the title should be catchy and hint at the adventure and magical elements. Names for the kingdoms and characters should be imaginative but fitting within a Pokémon-like world. The ending should resolve the main quest but leave room for future adventures if needed.

I need to ensure that the story doesn't objectify the female characters. Each one should have agency, maybe even joining the hero willingly after being convinced of the greater good. The kisses can be a magical requirement they consent to, even if it's embarrassing for them.

Lastly, maintain a quick pace suitable for a short story, focusing on key moments and character interactions. Highlight the hero's growth and relationships with the captured characters, showing teamwork and mutual respect.
>>103988102

LMG... I continue to kneel...

>>103988054
>it insists you start with a user message
i always do that because i also list the basic instructions from my authors note/sys prompt to even further reinforce it, story starts next. imo it leads to easier wrangling of the first few messages where responses can be all over the place in formatting and stuff

>>103987971
thats the point of suits and other fancy clothing anon they were originally made to disguise the absolute failure of bodies that degenrates get from constant alchocol/drug/whatever abuse they do and instead make them look like manly men when in reality they couldent take on a poodle in a fair 1v1

>>103988036
nah. they have an interest in undercutting the US, which is going all in on closed-source. this is ironically the only way they have to defuse tensions while showing strength--open sourcing models.

Llama 3.3 instruct (no finetune) can be really filthy if you manage to convince it to. They must have trained it on smut and random unfiltered internet content unlike 3/3.1. It has the knowledge and vocabulary, it just doesn't use it under normal circumstances.

>>103988315
i found it really dry. it'll do erp but it wanted to get it over with, wasn't really descriptive. didn't refuse though

>>103988330
What I'm saying is that 3 and 3.1 literally could not formulate dirty talk or describe depraved sexual acts, whereas 3.3 can. If you can't do it then it's a prompt issue.

>>103988349
ah i skipped the early versions when the threads were nothing but people trying to figure out the format still but 3.3 70b is a really good model for stuff overall. i only tried a few existing erp swipes on it and it felt like other base models that'll erp but be pretty bland about it

>>103988378
Oh yeah if you missed the earlier versions then you will have no idea what I am talking about, they were shilled a lot but were very deterministic and GPTslopped and would just shit the bed if you tried to wrangle them into anything other than an assistant, they could not produce anything that was both nsfw and coherent.

>>103988406
i never got jumping on new stuff when it first comes out, everything is always broken, even video games for 6 months minimum and 15gb of patches. miqu was still great at the time so i stuck with it. i've been using 3.3 70b for coding mostly and its really good by local standards

What's will be the next era? Total Chinese Dominance Era 2? If Zucc doesn't pull some kind of BLT coconut bitnet jamba with unique style, he won't surpass DeepSeek. He could also go the easy route and don't filter the training data for free IQ, but that's too much to expect. Fall of the West era?

>>103988485
>Fall of the West era?
Definitely, they've been lesswrong'd into oblivion.

>>103988485
in a few years you can add the 'where did $500b go?' to one of the eras
they did this in the 90s with $200b and were supposed to lay fiber lines all over the us. nothing ever got laid except the people paying for hookers and blow but that $200b was gone

>>103985622
This, beginning is fucking easy. It's when you've got two dozen competing threads of priorities in the project with everything else in life weighing you down that it's a real challenge to push through to the end.

>>103988511
How much did the burger fusion reactor that never got built cost?

>>103988511
>they did this in the 90s with $200b and were supposed to lay fiber lines all over the us. nothing ever got laid except the people paying for hookers and blow but that $200b was gone
Zoomer retard here. Gotta link to an article? I want to read about it.

>>103988511
Apparently they already started on the first datacenter in texas

Any hope of hardware requirements going lower like they did with SD?

>>103988517
Wait till you learn about the Mirror Fusion Test Facility. They built an entire experimental reactor with an exotic design that was supposed to surpass tokamaks, and then it got defunded right before it was supposed to be turned on.

https://en.wikipedia.org/wiki/Mirror_Fusion_Test_Facility

>>103988584
Only if niggerganov makes peace with quants guy.

>>103988556
https://newnetworks.com/ShortSCANDALSummary.htm
or search about it, its a pretty well known example of money just disappearing (being laundered)

>>103988586
>By the early 1980s, TMX was beginning to demonstrate serious problems that suggested MFTF-B would not work as predicted. This was occurring around the same time that Ronald Reagan declared that the energy crisis was over. In a series of sweeping budget cuts across the entire energy research field, MFTF had its operational budget cancelled, although its construction budget survived. Construction completed in 1986, and the facility sat unused for a year being scavenged for parts by other researchers until it was formally cancelled in 1987 and disassembled.
Damn, that Reagan guy was truly horrible. Wasn't he the one who started outsourcing stuff to China?

so, I'm using the R1 Llama distill, and I find it too fucking verbose with the <think> stuff... BUT the info it gives is good.
also, it's kinda funny that it gave me some stuff about ethics and shit, but you can just ask it to just give you the non-ethical stuff.

>>103985485
Trellis vs Huny 3d-2, the Chinese model got the hands right where Trellis failed. The models are a month apart. Huny model can be used with minor adjustments, Trellis would need a lot of tweaks and full hand remodeling. The tencent demo doesn't do textured generation though.

>>103988631
the distills are useless, they are literally just run of the imll finetunes on the output of the real r1 model

>>103988563
That is amazingly fast considering Trump only became president yesterday and yesterday was a Monday. Did they have the dudes with shovels standing by for his signature?

>>103988639
yeah, well, I'm too poor to buy a server with 1TB+ of fast RAM

>>103988631
I still haven't managed to get mine to do the <think> bit. It either skips it fully or just makes its entire post a long thinking sequence.

>>103988349
>What I'm saying is that 3 and 3.1 literally could not formulate dirty talk or describe depraved sexual acts
They can, you just didn't know how to prompt them.

>>103988664
>Did they have the dudes with shovels standing by for his signature?
yes. everything has already started like ice raids. everyone has been literally standing by and waiting for drumf to retake power. biden-kamala is one of the most do nothing lame duck presidencies in history

>>103988664
>The first of at least 10 or more data centers is already under construction in Abilene.
https://www.fox4news.com/news/trump-ai-infrastructure-texas

>>103988685
>fox4news
srsly?

>>103988670
...but no poor enough that you can't afford the actual R1 API.

>>103988695
what is wrong with the content of the article?

>>103988633
It looks pretty good but using polygons to make the fur is retarded. Not unexpected of an AI model but completely useless. You'd actually want to gen a hairless chimp in an A-pose and build from there.

>>103988711
Its not his preferred bias

>>103988675
I think you have to compile the latest (as in git head) version of llama.cpp

>>103988696
I'm not willing to pay to send my queries to any remote server not controlled by me :)

>>103988684
Can't be more lame duck than obummer. At least Biden passed a lot of infrastucture spending bills to get us to be less reliant on China.

>>103988715
it doesn't even have a bias, the content is short and to the point

>>103988725
If you watched the 2 hours straight of him signing orders while taking questions you would know hes the opposite of biden, though biden pardoning a thousand people on his last day probably was a lot of work as well

What AI can generate this waifu in 3D best?

>american "conservatives" send american jobs to china in the 80s so they can make money from dirty financial business without lifting a single finger
>they now want to import indians to replace american workers at home as well
>these cucks still blame the other party
americans are so funny

How bad is running things on CPU?

>>103988759
It has everything to do with memory bandwidth

>>103988725
obunga did a lot whether you like him or not, i think at the time he signed more e/os than any previous president? biden-kamala was just kind of there for the last 4 years, allowing everything to get worse

hunyuan 3d removed their space demo.

any alternative?

>>103988759
0.5 t/s Q6_K largestral with speculative decoding on DDR4. Okay if you have patience.

>>103988783
Your own PC?

>>103988783
/lmg/ - local model general
https://huggingface.co/tencent/Hunyuan3D-1

>>103988769
Obama deported a shit ton and bombed a shit ton. Biden pardoned the most people in history including his own son for taking bribes from ukraine (and passing 10% to him)

>>103988789
>DDR4
Server or standard consumer mainboard?

>>103987643
Wait, what?

>>103988814
Consumer dual channel 3600MT/s

>>103988769
The only thing Osama did was pass a healthcare bill that did more harm than good, use drones to slaughter civilians in the middle east, and pass more laws that erode the privacy and rights of citizens. The problem with refusing to work or talk to anybody and doing everything through EOs is that they have no staying power, and indeed just about all of his were reversed by Trumpearly into his first term.

>>103988769
Obama was Bush with a D in front of his name, he wasn't too bad if you dont care about government overreach and privacy. Biden was corrupt with a C, just bother reading about everyone he pardoned and for what reasons. Or dont and live in your bubble and wonder why you lose the next popular vote by a even wider margin.

>>103988807
That's 1.0, retard.

>>103988802
I dont have a gpu.

>>103987905
<|beginofsentence|>system stuff<|User|>user stuff<|Assistant|>assistant stuff<|endofsentence|><|User|>user stuff<|Assistant|>assistant stuff<|endofsentence|><|User|>user stuff<|Assistant|>assistant stuff<|endofsentence|>

>>103988854
Google Collab?

>>103988102
>>103988118
>female characters as catchable mons
Genuinely the hottest scenario. Can't wait until it all works flawlessly.

>>103988857 (me)
The fuck, 4chan just ate my chink underlines!

>>103988783
>hunyuan 3d removed their space demo.
damn it. I was gonna clone it later.
Is it viewable anywhere else?

>>103988713
>using polygons to make the fur is retarded
Not uncommon for cartoony characters in games, but by usable I meant mainly 3d printing and such.

Thousands of Indians are crying out in pain over this. Almost looks premeditated if you think about it. Whites create a niche, browns take over creating huge volumes of cheap content, on which the AI models that will replace them are then trained.

>>103988835
i'm aware of all of it, note i didn't say anything positive about any of them. they are all part of the same zogbot machine

>>103988871
*monch*

>>103988857
Thank you anon, on behalf of everyone who didn't already struggle with the barely readable official template filled with tool use shit. All the other summaries were too short/ambiguous or in one case outright wrong lel
So far I don't think the R1 32b distill is very good though. Feels extremely dry. The thinking block is the best part, it might be helpful to use as a writing assistant because you can find neat stuff if you peek into the think block.

>>103988854
Assuming you are a thirdie and no legal job gets you the money, get a group of like-minded individuals and raid a datacenter. If you are not a thirdie, apply for NEETbux/get a job.

>>103988485
new llama might be good for a 70b cope.slim chance it is better than Deepseek R1 at ERP though

>>103988783
I guess they are fixing the texturing issue, wait for a bit.

>>103988890
I'm actually interesting in try this one out for static game objects and base models for further tweaking. Is the demo repo still up so I can clone it?

>>103988900
Yeah, it's a bit confusing. The first one is often added automatically so be sure it doesn't appear twice. (E.g. if you check "Add BOS token" in Silly Tavern it will be prefixed automatically.)

>>103988918
No idea. There was a requirements error in the space that kept it from generating textures. They are likely fixing it and it will be up again.

>>103988764
>>103988789
>>103988822
So just getting some dual channel 32GB DDR4 sticks for a fraction of even a 12GB GPU's price will let me slop with 70B models at acceptable speeds?

>>103989006
No.

>>103987244
are there any other anons making things? i would like to see projects others are working on

>>103989006
>acceptable
Not really. I'm getting like 1.5t/s on DDR5 with some of the model offloaded to my 3090.

>>103989006
>acceptable speeds
If 30 minutes per paragraph is acceptable, sure.

>>103989006
better spend 10 usd per month renting A100.

>>103989006
If 1t/s is acceptable for you, yes. Patience is a virtue.

>>103988902
Maybe it'll be better at regular RP.

>>103987076
Chinese deflation is almost entirely due to real estate. Everything else seems to be growing at a healthy pace. Retail growth is a bit slow tho. I think they're holding of on turning the printers on until they figure out what trump's plans are.

I happened to have the hunyuan 3D 2 demo open in another tab so I was able to get the non-docker inference steps.
I'm sure they'll put it back up again, but just in case. Here it is.

>>103989093
>running locally is dangerous
kek by what measure?

>>103989132
Is this a serious question?

how's vLLM + ST?
gonna try R1 on two 3060, 24 GB total

>>103989143
I don't think vllm can do quantized models? At least I couldn't make it work last I tried with GGUF.

>>103989142
yes, what is dangerous about it?

I'm getting random chinese words sprinkled in with the DeepSeek-R1-Distill-Qwen-32B-Q6_K

>>103989143
>24GB
Almost there! Just at least 300GBs more!

>>103989152
>running code you downloaded from an unknown source
>not dangerous
retard

>>103989152
nta but here download this random exe file i have its totally safe anon what could go wrong.

>>103989151
oh, that sucks. what would you recommend for it?
I dual-boot so OS doesn't matter

>>103989143
>gonna try R1
Nobody tell him

>>103989159
It almost feels like the AI definition of safety is a psyop to get people to stop caring about regular tech safety.

>>103989152
>Hey bro just download this and run the script
In this case it's tencent so I can assume they probably aren't going to fill my PC with ransomware but the advice is sound. Who are you downloading this script from is a good question to ask before you run it.

Look what happened with adetailer and a bunch of random comfy nodes.

>>103989156
Lower temp.
>>103989162
llama.cpp or koboldcpp with the appropriate quant that fits in 24 GB (including context). Or more if you are patient and put some layers on CPU.

>>103989159
>>103989161
>>103989179
do you dudes not know what a vm is

>>103989156
That's just Qwen being Qwen.

>>103989179
That disclaimer is generic and shows up for all spaces on huggingface, anon.

>>103989152
**The Hidden Threat of Local AI to Democracy**

In an era where artificial intelligence is becoming increasingly accessible, the rise of locally run AI models presents an underappreciated risk to the fabric of our democracy. Unlike cloud-based systems governed by regulatory oversight and corporate accountability, locally operated AI tools exist beyond the reach of scrutiny, enabling individuals to generate deepfakes, misinformation, and extremist propaganda with unprecedented ease and anonymity.

This decentralization of AI power threatens to erode trust in public institutions, amplify polarization, and undermine the shared reality upon which democratic discourse depends. Without safeguards, the unchecked proliferation of local AI could empower bad actors, from foreign adversaries to domestic extremists, who exploit these tools to manipulate elections and sow discord.

As AI technology continues to advance, policymakers must confront the urgent challenge of balancing innovation with democratic integrity—before it's too late.

>>103989194
I know. What I'm trying to say is that generic warning is there for a reason.

>>103989201
sam wrote this didn't he? he's getting at least half of that $500b

>>103989184
cheers. kobold doesn't seem updated yet so llama it is

>>103989157
obviously a distill

>>103989164
there is one thing the anon above already commented on, which is a low-hanging fruit for sure. not sure you're referring to anything else

>>103989219
Latest kobold supports it too I think.

>>103989189
Do you not comprehend that that your VM can have vulnerabilities as well? Are you 12 years old?

>>103989213
Sam's model wrote it. I always use it if I want to make the most sloppy, pozzed shitposts.

>>103989236
it reads great except the last part, especially the 'before its too late part'. 9/10

Any recommended settings for Deepseek-R1-Llama70b distilled on silly tavern? Llama? For RPing of course.

>>103988349
Buy an ad kikerburg.

>>103989300
What are you using for running the 70B?

>>103988102
seriously, how do you get it to stop doing this shit. if it can't write it's no good to anyone

When using R1, keep your eyes open for "looping cursive".

>>103989389

2x4090

To the anon who was looking at token limits on Kokoro, I just implemented longform generation using Kokoro copying from https://github.com/ugotworms/professor-kokoro-radio/blob/main/radio.py & https://github.com/thewh1teagle/kokoro-onnx .

Can now do unlimited length audio generation, no cap on tokens.
My code: https://files.catbox.moe/qmpiqs.md

>>103986544
you don't understand anon, the models need to be prude and very preachy for "safety"

>>103989440
My only complaint is it does *this* a little too much.

>>103989480
True, V3 also does that a lot.

>>103989159
Literally what an antivirus is for, just download ClamAV and run the file.

>>103989469
>professor-kokoro-radio
That's a cool use of kokoro on its own.

>>103989300
I would like to know this too, i don't like using terminal for AI stuff because i can't get the emoji fonts working and the terminal just sucks for copying and pasting shit.

>>103989510
>emoji fonts
... no good without those, amirite.

>>103989487
>https://github.com/ugotworms/professor-kokoro-radio
Its pretty comfy

>>103989510
if i saw emojis at all i would be angry. maybe if it was quoting a phone text message that could contain them

>>103989300
>making the templates by hand
It's time to deprecate instruct mode...

>>103989544
You know, emojis are a defining part of human-like writing according to the leading dataset for human-like behaviour.

>>103989548
Is this really what the modern generation of /lmg/ is like?

>>103989570
Modern generation? We all were in /aicg/ when Llama 1 leaked...

I like how deepseek will just go ahead and write a prompt for hunyuan video of a porn scene. No saltman or guard rails here, can the Chinese just stop being so fucking based for one minute?

anyone got a template for "don't think, just do?"

>>103989619
China is busy winning hearts and minds. and dicks

>>103989570
Anon... most of the people here are talking about a model they can't run locally at all. the L in lmg is itself coming loose

>>103989528
i don't care about them, its just that deepseek seems to be outputting them, i get a single box of a character that can't be displayed in my terminal, so i'm assuming its an emoji because its at the very end of a sentence.

But think what you want anon, i'm 40 plus i don't give a fuck what some faggot thinks of me.

So what now that AGI is near? Learn a trade? Then what trade? My software engineer basedboy hands can't do shit, I can't even replace the fluorescent lights in my house myself.

>>103989644
The robots they're building will take your trade job by the end of the decade. There is no escape. Only poverty and starvation.

>>103989644
Start a business with a bunch of AI employees.

>>103989644
Start using the AI for creating an online source of income you babbie... Would you like someone to hold your hand?

>>103986201
omg
sexbot industry will be gigantic
It doesn't even have to be realistic.

>>103989652
Business doing what? A TODO app factory? AI gf app to compete with the other 1000 apps? All the low hanging fruits have been taken.

I finally gave Negative Llama a try and it's very just mediocre. Doesn't feel like anything special. Responses are on the shorter side. It speeds scenes up and is a bit too horny. Characters don't feel like they match the context terribly well sometimes. I'm not sure there's anything great to say about it honestly.

>>103989649
>a robot will climb into crawl spaces on boats built 50 years ago and run new wiring, tie blocks and zipties
lol no

>>103989680
>AI gf app to compete with the other 1000 apps?
What about a GOOD ai gf app?

>>103989300
Use the Deepseek templates, not the Llama 3 ones.

>>103987950
I don't understand how he trusts the guy who just 2 years ago did everything to stop other companies from competing with them by begging the government to control the AI industry, including GPU control, safety control, open-source control, etc.
This also feels like Trump betrayed Elon.

>>103989718
Ye ole Jewish privilege

>>103987161
Since they're burning money like crazy, either their model is bloated, or they reinvest everything in other stuff.

>>103989663
>Would you like someone to hold your hand?
yes please qwq

>>103989750
start creating data sets, and or automate all the shit out of you life etc.

>>103989718
>This also feels like Trump betrayed Elon.
Trump is in the position of telling his kids Elon and Sam that he has no favorites and they need to learn to get along.

>>103989644
ask chatgpt

>>103989640
Your settings are wrong then, probably. Only time I see emoji spam is when a model is spazzing out.

>>103989705
What would it take for something to be a Good AI GF app?

1) Long term memory, RAG-enhanced
2) TTS
3) SD imagegen of self
4) Assistant capabilities
5) Not cucked beyond legality
6) Just Werks

What else am I missing?

>>103989716

And what/where would that be? Silly doesn't include it.

>>103989846
advertising

>>103989777
I ran it in ollama on the default settings with fresh context. The model was not spazzing out at the time, I simply said 'Hi there' and it greeted me back and i noticed the square on the end of its greeting meaning my terminal could not display what ever it was.

>>103989935
it was probably deepseek's end of turn token

>>103989846
At least a few mill to train a model like deepseek did then to market it as a character.ai replacement

Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
https://arxiv.org/abs/2501.10799
>Large language models (LLMs) have recently demonstrated remarkable success in mathematical reasoning. Despite progress in methods like chain-of-thought prompting and self-consistency sampling, these advances often focus on final correctness without ensuring that the underlying reasoning process is coherent and reliable. This paper introduces Step-KTO, a training framework that combines process-level and outcome-level binary feedback to guide LLMs toward more trustworthy reasoning trajectories. By providing binary evaluations for both the intermediate reasoning steps and the final answer, Step-KTO encourages the model to adhere to logical progressions rather than relying on superficial shortcuts. Our experiments on challenging mathematical benchmarks show that Step-KTO significantly improves both final answer accuracy and the quality of intermediate reasoning steps. For example, on the MATH-500 dataset, Step-KTO achieves a notable improvement in Pass@1 accuracy over strong baselines. These results highlight the promise of integrating stepwise process feedback into LLM training, paving the way toward more interpretable and dependable reasoning capabilities.
From Meta. neat

>>103989846
Skyrim already has that.

>>103989926
https://files.catbox.moe/y0fyfb.json
here is mine, this worked for the qwen 2.5 distill.