SchwertImStein@lemmy.dbzer0.com to Programmer Humor@programming.devEnglish · 2 days agomuskrat's data eng expert's hard drive overheats while processing 60k rowslemmy.dbzer0.comimagemessage-square303fedilinkarrow-up1959arrow-down113
arrow-up1946arrow-down1imagemuskrat's data eng expert's hard drive overheats while processing 60k rowslemmy.dbzer0.comSchwertImStein@lemmy.dbzer0.com to Programmer Humor@programming.devEnglish · 2 days agomessage-square303fedilink
minus-squareeasily3667@lemmus.orglinkfedilinkEnglisharrow-up141·2 days ago60k is laughably, embarrassingly small. It’s still sqlite-sized.
minus-squarenaught@sh.itjust.workslinkfedilinkarrow-up57·2 days agoi mean its even excel sized depending on how many columns. This is seriously sad and alarming
minus-squareeasily3667@lemmus.orglinkfedilinkEnglisharrow-up11·edit-21 day agoHey now that’s real close to the 65,535 16-bit limit (from 20 years ago)
minus-squarewise_pancake@lemmy.calinkfedilinkarrow-up3·1 day agoHoly shit if this ids lm issue that’s too funny
minus-squaremorrowind@lemmy.mllinkfedilinkarrow-up49·2 days agoSqlite can easily handle millions of rows. Don’t sell it short
minus-squareKorhaka@sopuli.xyzlinkfedilinkEnglisharrow-up4·2 days agoI have an sqlite db that is a few GB in size, game saves using the format. Sadly almost all blob data, would love to play with it if it was a bit more readable
minus-squareThe_Decryptor@aussie.zonelinkfedilinkEnglisharrow-up8·1 day agoHow about a 6.4TB sqlite database? https://boyter.org/posts/searchcode-bigger-sqlite-than-you/
60k is laughably, embarrassingly small. It’s still sqlite-sized.
i mean its even excel sized depending on how many columns. This is seriously sad and alarming
Hey now that’s real close to the 65,535 16-bit limit (from 20 years ago)
Holy shit if this ids lm issue that’s too funny
60k is single json file
Sqlite can easily handle millions of rows. Don’t sell it short
I have an sqlite db that is a few GB in size, game saves using the format. Sadly almost all blob data, would love to play with it if it was a bit more readable
How about a 6.4TB sqlite database?
https://boyter.org/posts/searchcode-bigger-sqlite-than-you/
Should be enough to hold 60k rows
I’m not