July 11th, 2021 Newsletter

Lando

Lando

Statbot Developer

It’s been a long time since we published a newsletter; and with everything that has happened in the past month, it feels like just the right time to have one again!

Some important notes: Our newsletters will now be posted here on our Statbot blog. We will be doing newsletters separately from CBB, but be sure to check out CBB’s one here (link to newsletter blog entry for CBB) There will be no set timing for newsletters. When we have a few things gathered up for news, we’ll make one!

A Word From the Dev (The Mad Ramblings of a Shut-In)

A few of you know me as a loud, opinionated person who expresses frustration at minor things but then turns around and attempts to express level-headed arguments elsewhere. However, most of you probably just know me as "one of those Statbot developers". Or maybe you don't know at all and don't care... :(

For those who do care and/or are curious, I am the only developer who has been working on Statbot since the advent of v2 in early 2019 (which a vast majority of you know nothing about).

I don't say this to brag, but rather to lend itself to the following words. I inherited Statbot from Jonas back in 2018 at just 8k servers. That's already sizable for some developers, and for someone who did not know Javascript (all of Statbot is currently in Javascript), had no practical experience in programming, had no web design experience, etc., there was and has always been a lot to learn. But now I can say that almost 3 years later, Statbot is now something I can be very proud of. Recently, it has reached a state that I think is foundational for everything else to come which I'll be elaborating on.

Most of you don't need me to tell you that the bot up until very recently used to be plagued with stability issues which resulted in loss of tracking, and even the most recent disaster of lost data where we refunded premium no questions asked in response to the days of data lost. This is not even mentioning the other mishaps that resulted from novice navigation of cloud services, and their own buggy or unclear directions. However, two major things have happened in the past few months that make this a thing of the past:

  1. The website re-code was released in the beginning of June. This was a 9-month long process I undertook to rewrite the website from the ground up using everything I've learned over years + so much more I didn't know until I started this endeavor. Even now, every new feature I work on I learn something new about Javascript, or React, or Web Development, or Browsers, etc. Even after having created almost all of the dashboard by hand, I still rarely have all of the puzzle pieces I need to craft a new feature. Perhaps this results from me constantly pushing the boundaries of my knowledge; but either way, it's a time-consuming process. This consumed all of my time—that's why there was little in the way of updates outside of some bug fixes and troubleshooting stability issues during this time. However, I was able to push out the Dashboard Panels update in just a few days as a result of this work, and we'll all be continuing to reap the rewards in the near and distant futures.

  2. Statbot moved out of the cloud and is now running on our own dedicated hardware. It was a costly investment and it has been made possible with the support of everyone who gives back by purchasing a Premium upgrade. Seriously, thank you so much for putting up with the instability and sticking it out with us. While running our own hardware has put us at the mercy of our datacenter's network connectivity (we do have a backup plan if it becomes a recurring issue), this is cheaper, easier to scale given our available set of expertise, and removes yet another layer of abstraction (and sometimes absurdity) from the already complex operations of Statbot. This has also solved a weird bug in the bot that was causing something akin to a memory leak. Basically, every server would lose some minutes of stats every couple of days (if not a bit more frequently). A drop in the bucket for the most part, but incredibly important when boasting to be a source of truth on data. However, this would sometimes cascade and result in partial outages.

With the above factors in mind, I now have had the time to implement very detailed logging so that we now know when a shard outage happens almost immediately. Not only that, but I also tuned some settings of the bot using the new metrics. The recording functionality is literally (in the literal sense) the most stable it has EVER been, hence why I think this time is fundamental to a future filled with new features rather than me fighting fires trying to keep things running as they tear themselves apart. (I empathize with you devs out there dealing with that.)

I hope that gives a little visibility into the stat of the union. (Unintentional typo, but I'm keeping it.) I've been on break for the better part of June while I rebalance my life and meet with family I haven't been able to see since pre-COVID. Now I'm back and am already working on much needed NEW features and extended functionality!

PS: I'm thinking about possibly doing a "day with the dev" or something once a month (or two weeks or something) where I carve out some time to actually sit down and discuss the Statbot, roadmapped items, get feedback, etc. If you're at all interested in such a thing, let me know. No plans at all, just a thought I figured I'd toss into the wild for those that make it through this long post.

Overview of Site Revamp (Can I Get Some Patch Notes, Please?)

Above I talk about the "site recode"—how I have recreated the site from the ground-up at the code level to make it much easier to work in. However, there hasn't been a lot of information on what has changed and what to expect. Here I'll put down a list of the things that are most important to know:

  • Logging in to the site will now attempt to keep you on the page you're on. Before it would always send you to the Me page (now "My Settings").

  • There's a light/dark theme toggle :D

  • Graphs now show UTC for both hourly and daily data. This is primarily because users viewing daily data would see the time 0:00 UTC shifted to their timezone. This caused a lot of confusion about what time the data was showing. It's always been UTC, the graphs just now line up with the start of the UTC day rather than the hour offset shenanigans.

  • Very little else has changed to the Dashboard pages: Message, Voice, and Presence outside of the color changes and the Treemap for activities is now the bar chart that was on the old "Home" page.

  • The Overview page (previously "Home") now more accurately reflects the concept of a "dashboard", the previous design had aspects that were meant to be for searchability / SEO / recruitment through the Statbot dashboard (e.g. the invite link). This is not the current direction of Statbot and so related features / information have been moved or removed to focus more on the data.

  • The Overview page also is the first to have "Dashboard Panels". Once logged in, you can access this feature through the "Configure Panels" in the top-right of any dashboard's Overview page. Dashboard Panels are user- and server-specific. This means that everybody can create their own personal view. If you want to share configs, there are "Export" and "Import" functions available. Of the information that was removed from the default page, the number Member Count was such information. This information and much more is available in Dashboard Panels.

  • Channel Drilldown and Member Drilldown have merged into a single Premium upgrade. They have both been heavily updated to include things like adding multiple Users / Channels to the views, CSV exports across the site, and limitless Top Tables on the Message and Voice dashboard pages. There was already nothing like Statbot's Drilldowns available by any other bot, but that didn't stop me from making them even better.

  • You can copy-paste channel and member lists between filters on the site, such as from Statdocks to Statroles!

  • Member filters and Member Drilldown now allow you to search by username. Nickname will work in some cases, particularly in servers with less than 1,000 members. This is because of how Discord's (REST) API works. However, if you search by id and the user exists in Statbot systems, its username will show in future results.

  • Some channel management features in Settings (previously "Manage") have been deprecated. I am about to release a better version that allows aggregating old channels by name to clear up those filter lists and make things easier for people with ticketing systems.

  • Various stateful components will remember your settings on a given device. This includes tables with sorting headers, and the Heatmap toggle for servers with the Precision upgrade.