Post New Job

Jrmyprtr

Overview

  • Founded Date 11 11 月, 1966
  • Sectors 旅遊酒店
  • Posted Jobs 0
  • Viewed 6

Company Description

Open-R1: a Totally Open Reproduction Of DeepSeek-R1

Hey there! This post is an intro to the job, not a claim that we have actually replicated R1 yet. We’re integrating in the open, so as quickly as we have examination numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.

True, but it seems like there’s nothing to be evaluated since right now. I assume the supreme goal is to train a brand-new thinking model and after that utilize the same evaluation metrics as o1 and the DeepSeek-R1.

Well, there should be at least some sanity check and validation to guarantee the model was trained properly.

Oh yes, if you are speaking about the examination number of deepseek’s model it’s coming very quickly!

As pointed out in the blog post there is no model called Open-R1 to evaluate at all … not yet anyhow. This is a blog site describing that Hugging face will take the R1 Deepseek model, exercise how it was developed as laid out in the paper and from what they launched, and after that replicate that procedure.

in reality this is basically how science works … A creates a plan, discovery or development and it is checked by B, C and D to see if it is reproduceable. Thats been the foundation of research study now for a couple of centuries.

This blog site is not saying they have currently done so … Its a blog laying out an intent to start training a model like R1 and calling it Open-R1.

Also DeepSeek-R1 was just launched last week, and even in their paper they laid out the calculate hours needed. While those are low calculate hours for a SOTA design this does not imply you can train said design in a week. I ‘d personally enjoy to be able to train a transformer design in a week, but we might require to wait a while for that level of calculate technology.

So there are no standards for a design that has not been constructed yet right? As described in the blog site, and once again in reply to your concern.

However fear not, there is a GitHub Repo currently and contributors (hell I may join myself), some prelim work done, and a strategy of attack. A good beginning position.

n
@edbeeching
has evaluated the released designs already

( src: https://x.com/edwardbeeching/status/1884273209136275742)

R1 just trained on o1 outputs, so jointly …/ s. This is what the AI czars are saying

Hi! This blog post is an intro to the task, not a claim that we’ve recreated R1 yet. We will absolutely share the missing piece when we have them, you can expect the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

That’s nice and important to comprehend this significant hype that lacks technical understanding and explanation. Science is about recreation, and if they declare to be open, let them fullfill the open part.

Please do release the training cost.

We will!

Excalidraw Hi n
@bojan2501
thanks, we will undoubtedly be striving to ensure this training recipe can work for little language models on customer hardware because not everyone has a cluster of H100s in your home:-RRB- The tool we used for the images was Excalidraw! https://excalidraw.com

looking forward to it! WTF are your speaking about?

must be a joke

It’s actually cool to see how the entire open source neighborhood comes together!

Ops …

5.5 M is number press reporter in the deepseekv3 tech report (simply the training, not the experiment afaik), for R1 tough to estimate tbh however much less than 5.5 M imo

Historically, they have actually never ever released code or datasets of their LLM training, so I wouldn’t expect this time to be various. If they would launch it that would be remarkable naturally!

Yes of course!

So essentially you’re asking to change existing censorship with another flavour of censorship?

The code for the models are inside the design repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py

Hello Team, I’m Ray Bernard, the author and developer of EQUATOR. My research study group will be dealing with a paper focused on replicating particular components of DeepSeek R1. Our aim is to recreate the cold start and provide your group with a dataset that consists of COT and other techniques to support these efforts. We like to contribute our work to help. Please let me know if you discover this beneficial. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/

Where is the examination numbers? without it you can’t call it recreation.

8 replies

True, however it appears like there’s absolutely nothing to be examined as of today. I assume the supreme goal is to train a new thinking design and then utilize the same examination metrics as o1 and the DeepSeek-R1.

That’s quite interesting, I was asking myself why the concerns the author exposed here are not being asked by others? I believe the work they have done is remarkable but at the exact same time I wonder why they would not put these missing pieces on if they are supposed to be completely open.
Why even without recreation and comprehension of the development they could affect so much the market in this way?

4 replies

Hi! This post is an introduction to the job, not a claim that we have actually reproduced R1 yet. We will completely share the missing out on piece when we have them, you can anticipate the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

Interesting read, and it is great that we see more effort into this instructions: more optimization and less strength.
Also question what tool did the author use for developing action diagram.

2 replies

Excalidraw I’m so grateful that effort like this already exist, I’m gon na try to contribute:-RRB- 1 reply

eagerly anticipating it! So racist articel

2 replies

WTF are your talking about?

Awesome to have this open recreation started!

For Step # 1 check out https://github.com/open-thoughts/open-thoughts!

https://x.com/ryanmart3n/status/1884284101265612856

Let’s do this thing!

1 reply

It’s really cool to see how the entire open source community comes together!

Does anyone understand the actual training cost of r1? I can’t discover it in the paper or the statement post. Is the 6M expense reported by media just the number drawn from v3’s training expense?

2 replies

Ops …

Has anybody asked the DeepSeek team to release their training information and code, or at least share them privately with an independent duplication job like this? Have they turned down such a request?

A devoted duplication depends on using the very same dataset and hyperparameters. Otherwise, any significant disparities with the released criteria would be tough to pin down-whether due to training data distinctions or the duplication approach itself.

1 reply

Historically, they have actually never launched code or datasets of their LLM training, so I would not anticipate this time to be various. If they would launch it that would be remarkable of course!

In the meantime we have to make best guess estimates and see if we can arrive ourselves.

You supply good duplication process of Deepseek reasoning training. I will attempt something similar to it.

This is actually great information, can we tweak with specific usage case when code is released?

1 reply

Yes naturally!

Please consider removing prejudiced, polluted or unaligned training data and make an effort to get rid of copyrighted works from the crawl from intake. This will make the design more functional. If you recycled anthropic curation checks, this might also assist, remove obviouslybiased data will likely add a lot of value. We don’t want another polluted, unaligned open source design, right? And no business would ever utilize deepseek or a model that recycles it, right?
We appreciate your work for the advantage of humankind, we hope.
Miike C from NJ

1 reply

So generally you’re asking to change existing censorship with another flavour of censorship?

Can’t wait! Hopefully the design will be uncensored however whatever you can do is alright! Love seeing open source building itself up. I’m not clever adequate to really assist but I can contribute support lol

Hello guys, I am even just looking for code for DeepSeek-V2, in order to totally understand multi-head hidden attention. You do not appear to have code in Hugging Face even for that. Or am I missing out on something? Don’t see anything in src/transformers/models. MLA is not properly explained in their paper, so it would be very important to have code for this.