Post New Job

Pythomation

Overview

  • Founded Date 8 6 月, 1973
  • Sectors 消費產品
  • Posted Jobs 0
  • Viewed 6

Company Description

Open-R1: a Completely Open Reproduction Of DeepSeek-R1

Hey there! This blog site post is an introduction to the job, not a claim that we have actually replicated R1 yet. We’re developing in the open, so as quickly as we have examination numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.

True, but it appears like there’s absolutely nothing to be evaluated as of right now. I presume the supreme objective is to train a new thinking model and then use the exact same assessment metrics as o1 and the DeepSeek-R1.

Well, there ought to be at least some peace of mind check and validation to make sure the design was trained correctly.

Oh yes, if you are speaking about the assessment variety of deepseek’s model it’s coming very soon!

As pointed out in the post there is no model called Open-R1 to evaluate at all … not yet anyhow. This is a blog laying out that Hugging face will take the R1 Deepseek model, work out how it was developed as described in the paper and from what they released, and after that reproduce that process.

in truth this is practically how science works … A develops a strategy, discovery or development and it is evaluated by B, C and D to see if it is reproduceable. Thats been the cornerstone of research now for a few centuries.

This blog is not saying they have already done so … Its a blog site detailing an intent to begin training a design like R1 and calling it Open-R1.

Also DeepSeek-R1 was just launched recently, and even in their paper they laid out the calculate hours needed. While those are low compute hours for a SOTA design this does not indicate you can train said model in a week. I ‘d personally love to be able to train a transformer model in a week, however we might need to wait a while for that level of compute innovation.

So there are no standards for a model that has not been built yet right? As described in the blog, and once again in reply to your question.

However fear not, there is a GitHub Repo already and factors (hell I may join myself), some prelim work done, and a strategy of attack. A great starting position.

n
@edbeeching
has actually evaluated the released models currently

( src: https://x.com/edwardbeeching/status/1884273209136275742)

R1 simply trained on o1 outputs, so jointly …/ s. This is what the new AI czars are saying

Hi! This article is an intro to the project, not a claim that we’ve reproduced R1 yet. We will completely share the missing piece when we have them, you can anticipate the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

That’s nice and important to understand this significant buzz that lacks technical understanding and description. Science has to do with reproduction, and if they claim to be open, let them fullfill the open part.

Please do release the training cost.

We will!

Excalidraw Hi n
@bojan2501
thanks, we will undoubtedly be working hard to make certain this training recipe can work for little language designs on consumer hardware because not everyone has a cluster of H100s in the house:-RRB- The tool we utilized for the images was Excalidraw! https://excalidraw.com

anticipating it! WTF are your speaking about?

need to be a joke

It’s actually cool to see how the whole open source neighborhood comes together!

Ops …

5.5 M is number reporter in the deepseekv3 tech report (just the training, not the experiment afaik), for R1 difficult to approximate tbh but much less than 5.5 M imo

Historically, they have never released code or datasets of their LLM training, so I would not anticipate this time to be different. If they would release it that would be amazing naturally!

Yes naturally!

So basically you’re asking to change existing censorship with another flavour of censorship?

The code for the designs are inside the model repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py

Hello Team, I’m Ray Bernard, the author and creator of EQUATOR. My research group will be working on a paper focused on reproducing certain components of DeepSeek R1. Our objective is to replicate the cold start and provide your group with a dataset that consists of COT and other methods to support these efforts. We like to contribute our work to assist. Please let me understand if you find this beneficial. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/

Where is the evaluation numbers? without it you can’t call it reproduction.

8 replies

True, however it appears like there’s nothing to be evaluated since today. I assume the ultimate goal is to train a new thinking design and after that utilize the very same examination metrics as o1 and the DeepSeek-R1.

That’s rather fascinating, I was asking myself why the concerns the author exposed here are not being asked by others? I believe the work they have done is unforgettable however at the very same time I question why they wouldn’t put these missing pieces on if they are supposed to be totally open.
Why even without reproduction and comprehension of the development they could impact so much the marketplace in this method?

4 replies

Hi! This post is an intro to the task, not a claim that we have actually replicated R1 yet. We will completely share the missing out on piece when we have them, you can expect the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

Interesting read, and it is good that we see more effort into this direction: more optimization and less strength.
Also wonder what tool did the author usage for producing step diagram.

2 replies

Excalidraw I’m so that effort like this currently exist, I’m gon na attempt to contribute:-RRB- 1 reply

anticipating it! So racist articel

2 replies

WTF are your speaking about?

Awesome to have this open recreation started!

For Step # 1 check out https://github.com/open-thoughts/open-thoughts!

https://x.com/ryanmart3n/status/1884284101265612856

Let’s do this thing!

1 reply

It’s really cool to see how the entire open source neighborhood comes together!

Does anyone understand the real training cost of r1? I can’t discover it in the paper or the announcement post. Is the 6M cost reported by media simply the number taken from v3’s training expense?

2 replies

Ops …

Has anybody asked the DeepSeek group to release their training data and code, or a minimum of share them independently with an independent duplication project like this? Have they rejected such a request?

A loyal replication depends upon using the very same dataset and hyperparameters. Otherwise, any significant discrepancies with the released criteria would be difficult to pin down-whether due to training information distinctions or the replication method itself.

1 reply

Historically, they have actually never ever released code or datasets of their LLM training, so I would not anticipate this time to be different. If they would release it that would be amazing naturally!

In the meantime we have to make finest guess quotes and see if we can arrive ourselves.

You supply excellent duplication process of Deepseek thinking training. I will try something similar to it.

This is truly great info, can we tweak with particular use case when code is released?

1 reply

Yes obviously!

Please think about eliminating biased, polluted or unaligned training information and make an effort to remove copyrighted works from the crawl from consumption. This will make the design more usable. If you reused anthropic curation checks, this might likewise help, get rid of obviouslybiased information will likely add a lot of worth. We don’t want another polluted, unaligned open source model, right? And no corporate would ever utilize deepseek or a model that reuses it, right?
We appreciate your work for the benefit of humankind, we hope.
Miike C from NJ

1 reply

So basically you’re asking to replace existing censorship with another flavour of censorship?

Can’t wait! Hopefully the design will be uncensored but whatever you can do is alright! Love seeing open source structure itself up. I’m not wise adequate to really help but I can contribute support lol

Hello guys, I am even simply searching for code for DeepSeek-V2, in order to completely comprehend multi-head hidden attention. You do not seem to have code in Hugging Face even for that. Or am I missing something? Don’t see anything in src/transformers/models. MLA is not effectively explained in their paper, so it would be very important to have code for this.