I’m thinking this makes fullt sense because distillation is only additive, not subtractive. So it does not remove knowledge (if we can define censorship as removal of knowledge).
Most censorship isn't "removal of knowledge" but "installation of behavior that prevents some knowledge from being revealed or applied in certain ways".
This behavior can, in turn, be transferred via distillation. But, evidently, financial domain wasn't entangled enough with the censorship behaviors for them to bleed through, in this case.
Consider that LLMs are trained on the corpus of the internet, and (simplifying) consequently give the average answer of the internet. If the desired answer of the censorer is contradictory to this, then it requires additional training data to get the model to act a certain way.
distillation doesnt add anything; all it's doing is reconfiguring some root weights that get drowned out by noisy training and/or datset issues. It strengthens commonalities.
Deepseek is, with difference, the most "Western" of Chinese models, so it's a bit perplexing that it was chosen to test this hypothesis.
I didn't run any benchmarks but I played around a little, and after getting around the API-level filter Deepseek V4's answers about "China-sensitive content" aren't any different from what I get from Claude and ChatGPT.
Abliterated models certainly have their uses but they're not the default choice for most users or enterprises, and thus not the versions of those models most would interact with.
So, there is no subliminal learning in this situation, under what conditions would we expect it. I find a transfer attack to be a bit far fetched but it’s definitely interesting.
If we trained from random initialisations on DeepSeek output (that didn’t explicitly contain the political questions) we would expect transfer? And if we fine tuned a model pretrained elsewhere on Deepseek output?
It's most likely to occur when distilling a Chinese model from a Chinese base. We plan to do compliance geometry analysis in the future to see what is structurally changing in the model when distillation causes it to start refusing or whitewashing.
Yes but in which jurisdiction could you publish it?
We roughly know what the hot topics are for the current models, but actually testing and ranking would break said censorship and thus would be hammered into the ground through cointelpro methods by all parties.
It would be nice to have a hypothetical small country where the internal censorship would be non aligned and insignificant enough that it wouldn't take away from the overall findings. But it doesn't exist.
I want some science based authority on the moon where only 3-sigma IQ international academics have ultimate authority.
Oh wait Asimov did that right? I guess it didn't go so well either.
More important there are some things censored that are true. And some things censored that are false. How do we even get to a good model of the truthiness/nonsense adjustment indicator?
This behavior can, in turn, be transferred via distillation. But, evidently, financial domain wasn't entangled enough with the censorship behaviors for them to bleed through, in this case.
but there's no new information being created.
I didn't run any benchmarks but I played around a little, and after getting around the API-level filter Deepseek V4's answers about "China-sensitive content" aren't any different from what I get from Claude and ChatGPT.
We found V4 Flash was significantly more censored than the baseline.
ex: https://huggingface.co/huihui-ai/models
If we trained from random initialisations on DeepSeek output (that didn’t explicitly contain the political questions) we would expect transfer? And if we fine tuned a model pretrained elsewhere on Deepseek output?
What is the line?
It would be nice to have a hypothetical small country where the internal censorship would be non aligned and insignificant enough that it wouldn't take away from the overall findings. But it doesn't exist.
I want some science based authority on the moon where only 3-sigma IQ international academics have ultimate authority. Oh wait Asimov did that right? I guess it didn't go so well either.
More important there are some things censored that are true. And some things censored that are false. How do we even get to a good model of the truthiness/nonsense adjustment indicator?