BG = “#fafaf8” DARK = “#1a1a1a” # Color ramp: blue for common tokens, red for rare TOKEN_COLORS = [“#1a5276”, “#2471a3”, “#5dade2”, “#e67e22”, “#c0392b”, “#7d2a2a”] steps = np.arange(N_STEPS) fig = plt.figure(figsize=(16, 11), facecolor=BG) fig.suptitle( “SGD vs. Adam on Rare Tokens — Frequency Bias and Variance Normalization”, fontsize=14, fontweight=”bold”, color=DARK, y=0.99 ) gs = gridspec.GridSpec(2, 3, figure=fig,…
Anthropic announced Monday it has acquired Stainless, a startup founded by former Stripe engineer Alex Rattray whose software is widely used by rival AI labs, including OpenAI and Google. Anthropic didn’t disclose terms of the deal. However, The Information reported last week that Anthropic was in talks to acquire Stainless, which is backed by Sequoia…
Elon Musk suffered the worst defeat possible in his legal battle against OpenAI as a federal jury and a judge ruled he waited too long to bring his claims against the AI startup and its top executives, Sam Altman and Greg Brockman. The jury’s decision was a nonbinding recommendation sent to US district judge Yvonne…
New York public health provider NYC Health and Hospitals says a months-long data breach that allowed hackers to steal personal data, medical records, and fingerprints scans affects at least 1.8 million people. NYCHHC is the largest public health system in the United States and provides healthcare to over a million New Yorkers, the majority of…
The market for AI notetaking devices has exploded in the U.S., with the category generating over $600 million in revenue last year, according to a Menlo Ventures report. And as startups like Heidi Health and Freed, have shown, there’s decent demand for this tech in healthcare, where doctors and clinics see the potential for an…
Amazon announced the latest update to Alexa+ on Monday: the ability to generate podcast episodes on demand. The new feature, called “Alexa Podcasts,” is rolling out to customers in the U.S. today. Amazon describes the capability as a way to “turn any topic you’re curious about into a podcast episode, ready in minutes.” To use…
Grafana Labs, the maker of its eponymous fashionable open supply internet visualization software program, confirmed it had been hacked however that it refused to pay the hackers who had threatened to launch the corporate’s codebase. In a series of posts on social media, the lab stated its investigation discovered that the hackers had abused a…
Imagine you’re riding a motorcycle at 160 kilometers per hour when an arrow appears, floating on the road ahead, telling you exactly where to turn. No phone, no dashboard. Just your helmet, and a lens the size of a thumbnail. This is not a concept video. It’s heading to European roads as early as this…
What my mom lacks in healthy legs, she makes up for in a Claude Pro subscription. Having needled her repeatedly over the past couple years about AI’s environmental, political, and economic implications, I brushed all that aside on a recent Sunday and drove to her house. After a little tibia talk, I opened her computer…
Pretraining frontier-scale LLMs in FP8 is now standard practice, but moving to 4-bit floating point has remained an open research problem because narrower formats compress dynamic range and amplify quantization error at long token horizons. A new research from NVIDIA describes a pretraining methodology built around NVFP4, a 4-bit microscaling format supported natively by Blackwell…
