Version 3 Release Announcement - Sept 30th, 2022

Rejoice for Statbot Version 3.0 is upon us! Well, it has been for a month, but now you get to read all about it!

Here is the definitive source on all the changes that mark this pivotal moment in Statbot's history.

It's (About) Time

Has it felt like a long year for you? Statbot not really doing much, just kind of being there? I understand. Though for me it has been 10 long months of rewriting the entirety of the bot from the ground up to prepare it for this next phase in its development. But trust me when I say it has been worth it!

I'm pleased, a little relieved, and slightly anxious to finally announce that we can now call Statbot V3 "officially" released. Sure, it's been technically out for a month now, and several features have been available in beta for almost a year, BUT it's TODAY that will go down in the history books as the day Lando finally left his developer cave to actually share with you all the changes that he pushed out a month ago!

Admittedly, this important work has left me quiet as I focused on the enormous amounts of learning and experimenting that came with redesigning systems to use a new database tech and new frameworks. Due to the scope and impact of the pieces we're discussing (e.g., the database), it was difficult for me to find a way to handle this update incrementally. It was either months of two different data sources feeding the bot, dashboard, and other features with moments of prolonged downtime, or swap it all at once. I opted for the latter since keeping data tracking is the most important thing to all of us.

That left me toiling behind the scenes in a position where I couldn't work on many requests for new features and improvements because everything was about to change. But it's no longer that everything will change, it's everything has changed, and now we can move forward into a future with new and improved features like never before!

But before we talk about things we'd like to see in the future, let me go over the much-needed information on where we are today in terms of features. (You can read about the last month below under V3 Lore.)

The Changes

The question of "What changed in V3?" is a loaded, but I'll do my best to recall a year's worth of work so that you are able to use all the neat new things to your benefit and make requests for whatever else you may find yourself wanting.

General Changes

• The very foundation of Statbot itself — Seriously. I’ve rewritten everything in the past 12 months. Other than parts of the website, it’s all 100% completely brand-new code. I even redesigned most of the database.

• Gas, Gas, Gas — Statbot is now really fast. Large servers may not notice it on the user command for reasons I'll get into later, but everyone should experience huge performance boosts on the dashboard, and I will be even further optimizing systems as time permits.

• Time zones — Coming from beta, time zone handling is now fully-integrated into Statbot. Time zones are part of a server's settings and will affect the dashboard and commands. Both places display the time zone. (At the top on the dashboard, and the bottom of commands.)

• RBAC (Role-Based Access Control) Overhaul — Statbot's old permissions system made it difficult to see which roles had which permissions. The new permissions system mirrors Discord's and involves simply adding permissions and overrides to roles from a single menu.

• Forums Tracking — A very recent addition to Discord and already available in Statbot, Statbot tracks messages made in Forums channels. Statbot currently treats Forums Posts like threads, attributing the messages to the parent channel. I look forward to hearing suggestions to how to improve this as communities learn more about the roles they play in their communities!

• Voice and Activity Hours — All time values in tables and graphs now display as hours instead of the minutes. Over the years, some of you have mentioned the good point that large time values are conventionally displayed as hours on the internet. This was a common request, and now it’s commonplace. *Note at the bottom.

• Ephemeral Responses (User Setting) — Users now have the option to set the bot to respond to them using ephemeral messages. This allows users to view their stats in private if they wish to. This setting is currently only available through the dashboard at time of writing but it will make its way to the bot soon.

*Regarding minutes vs. hours. I have already seen some recent arguments for the displaying minutes as well, particularly since some other Statbot systems use minutes, and it wouldn't make a lot of sense to switch those to hours. Considering this, expect a new server setting in the near future.

Premium Changes

• Hourly data is now accessible for FREE
• The functionality previously allowed by Real-Time Caching for Statroles is now FREE and available to all servers
• The Statroles+ upgrade gained a new Condition type: Compare
• The Drilldowns upgrade gained a new dashboard page: Analytics
• Servers with History can now dynamically change Lookback on commands
• The Drilldowns upgrade heatmap on the Status now averages the data over the hours and days of the week making it much easier to see when peak hours are for online members
• Activity and Activity Type Statdocks will now part of the Data+ upgrade when they are re-enabled

Dashboard Changes

• Dynamic Query Filter Bars — Previously only available for the Drilldowns, you can now change your queries on the fly, share them by copy-pasting the URL, and even now save them as new Filter Presets to be used later by you and your team. Up until now, the dashboard was limited to a server's settings, much like bot commands. The Drilldowns were the first to break into customizing filters on the fly; but I felt that wasn’t enough, so I've standardized the entire dashboard to use a new, more powerful query interface found at the top of every page.

  • A notable mention here is the Filter Presets permission in the new RBAC system. This has the possibility of letting you set a role as being able to Read or Modify Filter Presets to give you finer control over access to settings and potentially sensitive information.

• Table searching — A new method of searching most tables on the site is now accessible through the icon in the top-left of supported tables. I created this because a necessary performance optimization I made a long time ago broke certain functions like ctrl + f (find on page) for tables. I had created a simple search bar, but it only supported searching the visible text in the table. This new search adds the ability to search things not even on the page, such as Discord IDs, and further adds more options such as numeric comparisons and “not” matching.

• Enhanced Graphs — Visible on the Messages, Voice, and new Analytics pages, I’ve introduced a new type of graph I created called an "Analytics Plot". These show you more than just the counts of a single series, allowing you to view multiple items at the same time.

  • Viewing multiple series on the same graph can emphasize correlation in data, such as when high message count corresponds with high unique members.
  • Now available on the dashboard are Unique Members and Unique Channels. Not just in total, but even as their own time series on the graph.
  • Analytics Plots support adding custom Moving Averages, a common tool in data analysis. These are lines that average the last N data points, so they show what the average of N was on any given hour/day/week/month. A way to interpret a Moving Average is how the averages in the data are changing, smoothing out the data. This reveals trends in particularly erratic data, which message and voice data tend to be.

• Analytics Page (Drilldowns Upgrade) — Being the namesake for the new plot type, the Analytics Page is the new frontier of the dashboard. I created it to handle most "single pane of glass" needs. Member and Channel Drilldown offer a fantastic way to see customized data for specific members and channels, but they lack a way to get a grand overview of the whole server. Whereas the Drilldowns compare the members or channels you know you are interested in, the Analytics page gives you a way to find those members and channels in the sea of data.

  • This page shows all Message, Voice, Unique Members, and Unique Channels at the same time. This has the added effect of allowing you to see the Unique Members / Channels across message and voice data combined.
  • This page uses the Filter Bar to its fullest potential, letting you quickly tailor the view to find the information you need. This even includes Message and Voice stat types as selectable values—useful for when you are not interested in them one of them influencing the unique values.
  • Just like with the member and channel tables on other pages, you can select Members and Channels from the table and quickly pivot with them to the relevant Drilldown.
  • Another new feature unique to this page is the "Not in server" metric—denoted as a red bar displayed on messages and voice stats. This is as it says, the number of message or voice activity that was from members who are no longer in your server. Note that this metric is not visible with a Role Filter in the query because that filter inherently only looks at data of members currently in the server.

• Aggregate Channel Update — With the advent of TiV (text-in-voice), Aggregate Channels needed a small redesign.

  • Aggregate Channels no longer have a type. Before, you would select which type an Aggregate Channel was— text or voice— and then merge Discord channels of that same type into it. Now, you choose which type of stats you want to merge into them when you go to create a merge task. This lets you merge message and voices stats from TiV's into different Aggregate Channels (such as if you have separate message and voice Aggregate Channels).
  • Deletion of old channels is now optional. You will generally want to keep this "ON" since many old channels can harm the bot's performance for your servers. Also, channels with stats still in them after the merge are ignored so there's no worry of accidentally deleting a TiV channel before you had the chance to merge the messages and voice activity into different channels.
  • Merges are no longer limited to only deleted channels. However, an existing channel cannot be deleted from Statbot's systems.
  • Aggregate Channels can no longer be deleted if they have any stats in them. However, they can still be merged into other Aggregate Channels if necessary and can be deleted along with that process.

Bot Changes

Everything. This is kind of a joke, but anybody using Statbot for more than a month will know it's also the truth. Literally everything about the bot changed, so I'll mention highlights and design choices that will be impactful going into the future. The Design Philosophy at the bottom of this page underpins these changes.

To clarify, when I say "the bot", I'm referring to the part that you interact with in Discord. i.e., The system that responds to Discord commands.

• The Code — Did I mention I rewrote everything? Well, I made sure to do so in a way that makes fixing, improving, and extending functionality MUCH easier. This will greatly accelerate iteration on ideas which will lend itself to many small, gradual updates rather than this all-at-once shenanigans you're probably used to with me by now.

• Slash Commands — These are a thing now. So that's cool. I'll mention again that all the regular Statbot permissions are enforced so there's no need to worry about misconfigurations in Slash commands letting users do things they're not allowed to.

• Components — In line with the design philosophy changes is the use of components to navigate around commands. I'm looking forward to making use of components to do even more so that you needn't run any more commands than necessary.

  • Commands showing stats have Lookback and Refresh options on them. These are just the first of the very flexible custom options settings I will be adding in the near future. Expect other settings like customized chart types! This is also a wonderful place to make suggestions. Let me know what kind of options you would like to see!

New Commands

• guide — This is the place to start for everyone new to Statbot, and even seasoned veterans of stats. I have plans to integrate more guides to provide condensed information of features directly within Discord. Try it now or after you're done reading this post!

• chart — This brings detailed charts for all stats into Discord. It's the new home for the labeled charts previously seen in the messages and voice commands when specifying a user, and also the member and channel commands.

  • Additionally, there are buttons to make it easy to navigate between stats and charts (available for the overlapping subcommands: messages, voice, activity, users, and channels).

• info — Previously show, this is the new home for viewing the activities a user is interacting with, getting a list of members in a role, and retrieving the Discord IDs of various things. This command will see more information added to it as the need arises.

• diag — This command helps you find permissions errors and the misconfigurations in your server. I'll be extending this command to make it easier to check permissions on specific channels and members when troubleshooting Statbot features.

Noteworthy Changes

• top — I have massively improved this command to now allow you to now page through the whole list of members and channels just like the dashboard. There is a limit of 250 members for free servers, with no limit with the Drilldowns upgrade just like the dashboard.

• settings — This command now presents a menu that allows you to navigate back-and-forth between the settings. Also, the new RBAC settings are accessible through this command marking the start of me being better about making sure the bot and the website are in sync.

• counters — The new place to create and now manage Statdocks. Here you will have access to many more options and settings through the bot than before, and all with just the click of a few buttons. You'll also be able to now manage your Statdocks through the bot. The Dashboard still remains superior in terms of selection, but this should make it much easier to explore and configure most of the options.

WIP (Work in Progress)

Some final notes on things that are in the pipeline that you will see within the next month.

• Activity and Activity Type Statdock counters are still disabled at the moment. I’ll be working to get them back online in the next couple of weeks.

• This documentation and relevant information on the website need some TLC. They will get it.

• Statbot has been tracking member Joins and Leaves for the last month, I just need to surface the data so expect these to be coming soon to Members graphs near you!

• Localization for bot commands and responses are mostly done. There is just some remaining legwork involved before we can begin translation efforts.

• The help command will improve to include navigation and examples.

• Despite and in part because of performance optimizations, the user command is slow in servers with many thousands of active users. This is because of the new "Rank" part of the command is extremely computationally heavy as it must compute and sort increasingly active users. This would have been completely impossible to implement without the performance optimizations, hence why they are partly to blame. I will be looking into how to improve this to return the command to a state even better than its earlier glory!

Additional Information

Sale!? Sale!

Hey, did you know a sale has been going on for the last month? Yeah, me neither, but we do now! No promo code necessary, just go to the upgrades page and enjoy an instant 40% off with an additional 15% available during checkout. We'll be running this sale for the next two weeks, ending on October 16th, so be sure to claim it while you still can!

Feature Requests

As I said at the beginning of this post, I am now ready to consider feature requests more deeply. Coincidentally, Discord's new Forum channels are the perfect way to collect and respond to such requests and so there is a new #feature-requests Forum channel in the support server. I will now be seriously looking over and see how quickly I can solve your needs, all that I ask is that you be clear and detailed in what your requests.

Remember, the more context you can supply me about the problem you need solved, that better I can try to solve that problem. The Guidelines in the channel will help you understand more about how to do this so do please read them before making a request. Remember that it’s not a negative thing if you are asked to elaborate, provide more context, or otherwise explain a little more about your request. This is just part of the process of getting you the tools you need.

Events

I'm playing around with the idea of hosting events in the server, mostly around the concept of dev talks where I walk through new features, gather feedback, and ultimately get better about making sure I'm doing my part in listening to your evolving needs. The recent silence on my part has been necessary to get this far and it's not as necessary anymore. I don't have any concrete plans at the moment, just don't be surprised if you suddenly see an event scheduled in the Statbot server.

Community Builders

Lela will make an official announcement about this soon, however it is worth mentioning that we will be reintegrating the Community Builders part of Statbot's operations back into the support server. This will make it easier for everyone to join in on the conversations, as well as make it easier for me to see other kinds of problems that Statbot may be able to aid in solving.

Boring Stuff

Here's the lengthy things that I wanted to say but it doesn't really add much to the idea of making sure you knew what was in the update, so I put it all down here for those who are curious. Enjoy.

V3 Lore

First, I understand that V3 just kind of, happened, and that it has been continuing to "kind of happen" over the last 4 weeks. The only people knew that there was something this big coming down the pipeline were anybody who actually believed my notes in prior announcements about "big changes coming" and Premium members who had had the chance to beta test and influence the new direction of bot commands. However, nobody knew what September 1st had in store for us, not even me.

Long story short, the biggest part of the V3 update was redesigning Statbot's very foundation: the database and the logging. These mission critical pieces were approaching critical mass where operation was becoming unstable from time to time. The database was slowing down for servers with years of data, and the logging for the bot was outgrowing their pretty beefy servers and pushing that iteration of Paracord to its limits.

This work began many months ago last November. It began as some simple tests using a new database tech. But after I learned what it could truly do, I rushed to rewrite the bot. The problem with this is that redesigning the very foundation of Statbot itself meant redesigning everything. And so that's what I've been doing all the way up to today. Though of course, if I was going to redesign things, I wasn't going to hold any punches.

An outage on September 1st was an instance where the loggers started having a fit and for the first time, I was unable to get them to behave in a reasonable time=frame. After exhausting my options, I decided to pull the cord. If I couldn't trust the old system to remain stable anymore, it was time to switch things over.

Therefore, tracking was not affected. The new systems were ready and had already been tracking for a couple of months, I just needed to migrate the data. However, many of the other components such as the dashboard, commands, and Statdocks had not yet been integrated full into these new systems. That's what I have spent 12+ hour days working on the past month.

Though in truth, the 12-hour days are only beginning for me, and they may never stop. The only difference between today and yesterday is that I am declaring that V3 is officially released and reopening things like Feature Requests. What today truly marks is my renewed commitment to adding features that you need to manage your communities.

Why "V3"?

I've mentioned before how versioning is not really my strong suit so the version itself is not super technically important. However, this marks a shift in design philosophy down to the very core of Statbot, similar to the V2 update. It's because of this that this update is worthy of being called the next official iteration of Statbot. V2 came with a set of limitations that V3 make entirely trivial. V2 had a codebase that, albeit easier to work with than V1, pales in comparison to the engineering and ease of development within V3's ecosystem. V2 extended the website and bot's functions greatly to provide more ways of viewing data and V3 takes all of that to the next level. V2 introduced many performance optimizations, some of which did not stand the test of time, while V3 tunes the heck out of the bot with 3 years of learning about Statbot's specific bottlenecks to back it up.

V3 Philosophy

Let's start with the context of these changes and through which everything else will be applied through. My design philosophy underpinning all of the bot's new features is the idea of "Flow".

To me, Flow is that a new user shouldn't have to know how to get to a place to get there. Indeed, they should be able to merely know what they want, and then they should get it with the click of a few buttons. However, this should not be at the sacrifice of inconveniencing users who know what they want and where it is.

It's a balancing act and one that I'm committed to perfecting with your feedback.

Here are some of the things I've implemented towards this end:

• Components and Interactions — This system has received a lot of attention and care from Discord since its inception back years ago and I agree that it's the way forward. The bot seeks to squeeze every ounce of utility out of these to provide never-before-seen customization and navigation.

• Slash / Prefix Commands — With Slash commands now available in the bot, I should mention that there's no intention to remove Prefix commands. The two different ways of calling commands feed into the exact same back end, meaning that commands supported in one will be instantly available to the other. The only difference will at times be the available options since Interactions and Components have limitations, and text input itself has its own challenges.

• Function AND Form — Statbot's previous command responses often got the job done to an extent, but "to an extent" is not good enough. While some of you may not have been bothered by how it visually presented data, visual presentation does affect others' ability to take in the information. All of us have a limit where the signal gets too strong and overwhelms us—where the amount of data, no matter how useful, become useless because we're overloaded. If Statbot has never felt that way for you then great! For everyone else, I wanted to accommodate them by easing the visual strain and clutter of the bot.

• Inferencing & Searching — This mostly applies to prefix commands and text input, but is important to me all the same. Part of the new system involves the bot inferring what you meant to do or are looking for. A couple of examples are the new filter command and how the bot responds if you type a Prefix command. Likewise, if you are attempting to search for a user or channel that may no longer be in the server (and so not accessible as slash command options). I hope to incorporate inferencing in the bot more

---

Keep up-to-date on Statbot changes by subscribing to the @Bot Updates role in the Statbot support server!