Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
AbstractPhil 
posted an update about 11 hours ago
Post
36
Due to the licensing issues and changes to Anthropic terms of service, I had to rebuild the core dataset rows with GPT, Deepseek, and Gemini. A week of prep and grappling with dataset logistics later, we have enough candidate rows to do a preliminary train set.

Thank you for your patience as I sent the required requests to Anthropic for the original generations from Sonnet 5.5 in conjunction with Opus 5.5 reviewer. The original 1.0 series must be respected as non-commercial, and in this case I'm barring them from the upcoming trainings. There are about 7000 publicly visible and 8900 or so total rows.

The new set is around 11500 total prepared from Deepseek flash 4.1, GPT 6.1 Luna, Gemini Argon then Carbon, and GPT Astra med.

In comparison to the original Lucia, it was prepared using only around 1900 rows and very little to no regularization. This is meant to be heavily regularized and prepared with many additional rows from multiple sources.

We have three series of row sets that MUST be respected, as the original Lucia line is 100% non-commercial usage until the approval or denial from Anthropic.

---
For the upcoming trains I will be training two directly different models. The non-commercial variant will have all of the rows, while the apache-2.0 variant will have only the apache friendly prepared rows.

One is Lucia, the Apache-2.0 variant. Fair use and structurally viable for use-cases as per the licenses applied from the datasets. This model is meant to be provided as-is. Datasets such as coco 2017 are represented in this set. Any rows requested for takedown will need to be respected and will be moved to the non-commercial set.

The other is non-commercial Lucia v1.1. This variant is meant to be used for research only purposes. This has been trained with a wider diversity of rows, specifically sourced from non-commercial complex image sets like DOCCI and Deepfashion.

These two model lines will be experimentally built by Thursday.
AbstractPhil
•
edited about 9 hours ago

These reviewers were not allowed to modify rows. Their entire job was to approve, mark fixes based on flags, or mark rejected.

The reviewers for the 11,000 rows were;
GPT Astra 6.0 High
Claude Fable 5.1 - Max
Claude Fable 5.2 - Max
Claude Opus 5.5 - Max
Claude Sonnet 5.5 Agents - Max
Claude Haiku 5.5 Agents - Max
Gemini 4 Argon - High
Gemini 4 Carbon - High

The API interfaces consulted for DIRECT row generations were;

These models were sent the requests and images to synthesize the rows. The requests would often include a list of pre-synthesized prompts and tags for assistance. Many were sent images, many were sent no images.

GPT Luna 6.1
Deepseek 4.1? Flash
Gemini 4 Argon
Gemini 3.8 Flash

Claude was omitted from generations, while review cases were specifically handled hands-off. Meaning ONLY OPINION based on accuracy without any direct generation input was used.

This is how the generation system rules were respected.

Arbitrations for approval passed up the chain appropriately via pecking order.

Multiple Haiku models were used for adversarial approval. If there were disputes, Sonnet would handle them. If Sonnet was uncertain, Fable max was handed them.

If Fable was uncertain I would handle them.

Astra max approvals were considered accurate enough to trust almost every time.

Argon-low approvals went up the Haiku chain for post-review analysis, if there were disputes they were sent back. Argon-High were effective reviewers.

I ran out of weekly usage before I got a full benchmark from Carbon, but the early results were promising.

Empirically, Gemini Argon was weak for row generations. The system could not create the necessary implications more often than not.

Deepseek was the weakest. The model barely handled even the more simple tasks, and often completely hallucinated invalid or incorrect results.

GPT 6.1 Luna was very strong. Almost all the rows were within the rule parity, but they still required multiple sendbacks for many.
AbstractPhil
•
edited about 6 hours ago

After refining the process and the systems, we have managed to increase the row count from 11,500 in the last 8 hours to nearly 20,000.

This is due to Fable 5.2 taking the helm over Opus 5.5. Fable 5.2 is making suggestions related to structure that Opus 5.5 wasn't making. In conjunction with the two Haiku reviewers in working with the Sonnet arbitration giving more power to Sonnet and Astra, the system is performing increasingly more accurately.

Astra has also begun marking lighting and providing more than just notes on lighting. This will likely increase the accuracy and quality for the lighting predictions with extractions. Being a reviewer as well, this cuts into the Astra reviewer time, so increasing the throughput to the haiku pair was important.

With that I've reduced my API costs and increased the throughput, while still creating passable rows with a much more accurate outcome than with Gemini and Deepseek in the mix. Fable 5.2 randomly samples the images from the rejections, as well as randomly sampling from the accepted rows for arbitration. Almost all of the accepted rows sampled fit the rules and standards, so the majority of the rows will be very strong.
In this post