hyperparam @hyperparamapp

Using JavaScript to make better AI hyperparam.app Seattle Joined December 2023

Tweets

92
Followers

50
Following

84
Likes

55

Kenny Daniel @platypii

2 months ago

Just wrote a new blog post: how I spent a year making the world's fastest Parquet loader in JavaScript, and all the optimizations that went into it. TLDR: Hyparquet can load a parquet file from S3 in 155ms. While duckdb-wasm took 3466ms the same file!

2 5 31 2K 9

Download Image

hyperparam @hyperparamapp

2 months ago

"This involved collecting, curating, and training on high-quality math data" 👀

Noam Brown @polynoamial

2 months ago

"This involved collecting, curating, and training on high-quality math data" 👀

4 10 326 42K 26

0 0 2 25 0

Kenny Daniel @platypii

4 months ago

Shots fired at the @ApacheIceberg Meetup: "What I see is a community that's gotten way too complacent with complexity. I think there's a way you can do this a LOT more simply." ...so I ported Iceberg to JavaScript.

0 1 1 119 0

Download Video

hyperparam @hyperparamapp

4 months ago

Announcing the Hyperparam OSS Universe! We wrote a suite of open-source libraries for working with large AI Datasets (eg- parquet files) entirely in JavaScript. This is the way to build faster, more interactive, and more scalable data applications in the browser.

Kenny Daniel @platypii

4 months ago

1 0 10 2K 1

0 0 2 43 0

Kenny Daniel @platypii

5 months ago

duckdb is great! but a couple points for hyparquet: - duckdb wasm is ~40mb (often larger than the data being loaded) - bundling wasm is a pain - hyparquet is tiny (10kb) and easy to deploy - if you want to minimize time-to-displayed-data in the browser, hyparquet usually wins.

Alex Bezh 🇺🇦 @alexbezhan

5 months ago

0 0 0 358 0

1 1 5 324 0

hyperparam @hyperparamapp

5 months ago

OpenAI: A Lesson in How NOT to Write a Parquet File This week @OpenAI released new datasets on @huggingface -- great news for open data! But their specific parquet files makes me sad... They released a 300mb parquet file that contains 400 rows in a single parquet rowgroup. When…

hyperparam @hyperparamapp

5 months ago

0 0 0 430 0

0 1 5 521 1

Kenny Daniel @platypii

5 months ago

"before we were largely in a compute-constrained environment. now we are much more data-bound."

Sam Altman @sama

5 months ago

"before we were largely in a compute-constrained environment. now we are much more data-bound."

15 23 537 142K 106

Download Video

0 1 1 160 0

Kenny Daniel @platypii

5 months ago

Just found this very cool project OpenTimes where you pick a starting point, and it gives you travel time to all points around you. Uses hyparquet to efficiently fetch geospatial data and visualize it. I absolutely love this kind of browser engineering!