Glad to report that all issues have been completely found and fixed since yesterday. BRK is now fully operational and back to normal. While everything ran okay on a beefy-ish machine on internal storage (which is where BRK was running those last few days), running it on more common hardware and more particularly external storage became more challenging with all the latest features. This went unnoticed as the load was relatively low when added over time but running it fully, in one go, would end up in a failure. Several patches were applied such as converting low density datasets to lazy computation, IO optimizations and many others. While those changes helped, they did not solve the root cause of the problem. In the end what really worked was creating a little “database” where each vec/dataset/array is stored sequentially in a single memory mapped and sparse file for very fast read speeds instead of one file for each. Some new bugs were probably introduced but the service has been successfully running for more than 24h on both servers. Offering a good self-hosting experience will always be a very important goal of this project. https://brekit.org #Bitcoin #BRK

A secure/nostr version of https://hack.chat could be really interesting

Somewhere in this range sounds plausible, there are many silent stakers

True neutral

Finally getting somewhere It seems to be corrupted filesystem structure due to too many memory mapped files opened at the same time among other things Hardware is most likely fine ! Which is really good news for all self-hosters Have to clean everything, sync all nodes and try to rerun BRK with some changes and fixes, will take a little while but hopefully everything will get back to normal

I’m *now* paying

Very kind, I might take you up on it Will see how it goes ! Thank you

Yeah, everything worked for months but with more datasets added over time, I must’ve reached a threshold for which I’m not paying the price

And down again… Everything works properly with an internal SSD but something very wrong is happening with external SSDs Tried many things but no success yet. Hopefully will get to the bottom of it

The silver lining is there also a new version of BRK v0.0.75, which brings you an incredible amount of addresses related datasets/charts and many pillars to build the explorer. This brings BRK to close to 108 000 dataset variants ! With many more to come. Should be enough for most but if you're still missing something, just ask and we'll do our best to add it if possible Here's a quick peak: Check it out at: https://bitcoinresearchkit.org #Bitcoin #BRK

We are back online folks Apologies for the inconvenience Many lessons were learned during this event, a lot of work will be put to avoid it happening again.

Sorry everyone a reorg caused another bug which killed the last remaining server The other server has a faulty ssd thus can’t run either Working on fixing it all !