Add event environments and filtering

docs/docs/clients/dotnet/configuration.md

@@ -5,6 +5,8 @@ order: 1
 
 # Configuration
 
+Set a deployment environment once at startup, with optional per-event overrides. See [Environments](/docs/environments/) for configuration examples.
+
 There are a few ways to configure Exceptionless in your project. We'll cover them here or you can jump to app-specific examples: [Console App Example](/docs/clients/dotnet/guides/console-apps-example), [Web Server Example](/docs/clients/dotnet/guides/web-server-example).
 
 ---

docs/docs/clients/javascript/client-configuration.md

@@ -5,6 +5,8 @@ order: 1
 
 # Configuration
 
+Set a deployment environment once at startup, with optional per-event overrides. See [Environments](/docs/environments/) for configuration examples.
+
 - [Installation](#installation)
   - [Browser](#browser)
   - [Node.js](#nodejs)

docs/docs/environments.md

@@ -0,0 +1,69 @@
+---
+title: "Environments"
+---
+
+# Environments
+
+Use an environment to distinguish production, staging, development, or a custom deployment within one project. Each event can carry one optional top-level `environment` string. The same error still belongs to one stack, with one shared status and fixed version.
+
+## Configure your client
+
+Set a default during application startup. In the .NET client:
+
+```csharp
+client.Configuration.SetEnvironment("production");
+client.CreateLog("Deployment complete").SetEnvironment("staging").Submit();
+```
+
+You can also set `Exceptionless:Environment` in .NET configuration, or the `Exceptionless__Environment` environment variable. The hosting integration uses `IHostEnvironment.EnvironmentName` when no explicit environment is configured. A per-event value overrides the default.
+
+In the JavaScript client:
+
+```javascript
+await Exceptionless.startup((config) => {
+  config.apiKey = "API_KEY_HERE";
+  config.environment = "production";
+});
+
+await Exceptionless.createLog("Deployment complete")
+  .setEnvironment("staging")
+  .submit();
+```
+
+For direct API submissions:
+
+```json
+{
+  "type": "log",
+  "message": "Deployment complete",
+  "environment": "production"
+}
+```
+
+Names are trimmed, with a maximum of 64 characters. Events preserve the supplied casing: `" Production "` is stored and returned as `"Production"`. Filtering is case-insensitive, and aggregation keys are normalized to lowercase, so `Production` and `production` share one environment bucket. Custom names such as `qa-west` are supported. Empty, oversized, control-character, and non-string values are treated as unspecified. Historical events and older clients without this field remain unspecified; they are never assumed to be production.
+
+## Filter your data
+
+Choose **Manage filters → Environment** on Events, Stacks, Sessions, or Stream. Select one or more names, or **Unspecified** for events without an environment. Clear the selection to include all environments. The picker discovers names for the selected projects and time range; you can also enter a name that has no current events.
+
+Environment selections persist in URLs and saved views. Events, sessions, and stream tables have an optional Environment column. Event details show Environment in the overview; **Machine & runtime** contains the existing `data.@environment` diagnostics.
+
+Environment searches and aggregations are available on every plan:
+
+| Search | Meaning |
+| --- | --- |
+| `environment:production` | Production events |
+| `(environment:production OR environment:staging)` | Either deployment |
+| `_missing_:environment` | Historical or unspecified events |
+| `_exists_:environment` | Events with an environment |
+| `environment:"qa west"` | A custom name containing spaces |
+
+Stack dashboard counts and charts use events matching the filter. Stack status remains shared, so changing status while viewing production also changes the same stack seen in staging. Automatic session detection separates the same user's activity by environment.
+
+## Fixing stacks across deployments
+
+Use [fixed in version](/docs/versioning/) when environments run different releases. For example, if a stack is fixed in `2.4.0`, occurrences from production running `2.3.0` do not represent a regression just because staging already has the fix. Continue sending the application version with events. Plain **Fixed**, without a version, retains its existing behavior across all environments.
+
+## Rollout
+
+Deploy the server before adopting SDK versions that expose the new setting. The server adds mappings to existing event indices without rewriting historical events. Clients that do not send an environment continue to work. `data.@environment` and custom `data.environment` values retain their existing meanings. Root `environment` values are also preserved as legacy custom event data, including submitted key casing and the original whitespace in strings. Non-string values leave the deployment environment unspecified. If `data.environment` already exists, the root value uses the next available key (`environment1`, `environment2`, and so on).

docs/docs/filtering-and-searching.md

@@ -6,6 +6,7 @@ title: "Filtering & Searching"
 
 - [Filter by Organization \& Project](#filter-by-organization--project)
 - [Filter by Time Frame](#filter-by-time-frame)
+- [Filter by Environment](#filter-by-environment)
 - [Filter / Search by Specific Criteria](#filter--search-by-specific-criteria)
 - [Searchable Fields \& Requirements](#searchable-fields--requirements)
 - [Multiple Queries](#multiple-queries)
@@ -30,6 +31,10 @@ Click on the calendar icon in the header to select from multiple preset time fra
 
 ![Exceptionless Filter Time Frame](img/filter-by-timeframe.png)
 
+## Filter by Environment
+
+Choose **Manage filters → Environment** to select production, staging, development, or a custom name. Select **Unspecified** for events without this property, or clear the selection to include all environments. See [Environments](/docs/environments/) for client configuration and shared stack behavior.
+
 ## Filter / Search by Specific Criteria
 
 Click the magnifying glass to search by specific criteria.
@@ -53,6 +58,7 @@ View a complete list of searchable terms, examples, and FAQs below.
 | stack              | `stack:54d8315ce6bb2d0500bcc7b4`                                                                                     | true                         | Stack id                                      |
 | reference          | `reference:12345678`                                                                                                 | true                         | Reference id                                  |
 | session            | `session:12345678`                                                                                                   | true                         | Session id                                    |
+| environment        | `environment:production` or `_missing_:environment` | true | Deployment environment |
 | type               | `type:error`                                                                                                         | true                         | Event type                                    |
 | source             | `source:"my log source"` or `"my log source"`                                                                        | false                        | Event source                                  |
 | level              | `level:Error`                                                                                                        | true                         | Log level                                     |

docs/docs/versioning.md

@@ -4,6 +4,8 @@ title: "Versioning"
 
 # Versioning
 
+Stacks and their fixed version are shared across [environments](/docs/environments/). Continue sending application versions so older deployments can report occurrences without incorrectly reopening a stack fixed in a newer version.
+
 You can mark error stacks fixed and they won't show up or notify you until they regress!
 
 ## How does this work?

src/Exceptionless.Core/Extensions/PersistentEventExtensions.cs

@@ -183,6 +183,7 @@ public static PersistentEvent ToSessionStartEvent(this PersistentEvent source, I
     {
         var startEvent = new PersistentEvent
         {
+            Environment = source.Environment,
             Date = source.Date,
             Geo = source.Geo,
             OrganizationId = source.OrganizationId,

src/Exceptionless.Core/Jobs/EventPostsJob.cs

@@ -267,7 +267,9 @@ protected override async Task<JobResult> ProcessQueueEntryAsync(QueueEntryContex
                 AppDiagnostics.PostsParsingTime.Time(() =>
                 {
                     string input = encoding.GetString(uncompressedData);
-                    events = _eventParserPluginManager.ParseEvents(input, ep.ApiVersion, ep.UserAgent);
+                    events = ep.IsNormalized
+                        ? _eventParserPluginManager.ParseNormalizedEvents(input)
+                        : _eventParserPluginManager.ParseEvents(input, ep.ApiVersion, ep.UserAgent);
                     foreach (var ev in events)
                     {
                         ev.CreatedUtc = createdUtc;
@@ -310,6 +312,7 @@ private async Task RetryEventsAsync(List<PersistentEvent> eventsToRetry, EventPo
                 // Put this single event back into the queue so we can retry it separately.
                 await _eventPostService.EnqueueAsync(new EventPost(false)
                 {
+                    IsNormalized = true,
                     ApiVersion = ep.ApiVersion,
                     CharSet = ep.CharSet,
                     ClientKeyHash = ep.ClientKeyHash,

src/Exceptionless.Core/Migrations/010_AddEventEnvironment.cs

@@ -0,0 +1,36 @@
+using Exceptionless.Core.Extensions;
+using Exceptionless.Core.Models;
+using Exceptionless.Core.Repositories.Configuration;
+using Foundatio.Repositories.Elasticsearch.Extensions;
+using Foundatio.Repositories.Migrations;
+using Microsoft.Extensions.Logging;
+
+namespace Exceptionless.Core.Migrations;
+
+public sealed class AddEventEnvironment : MigrationBase
+{
+    private readonly ExceptionlessElasticConfiguration _configuration;
+
+    public AddEventEnvironment(ExceptionlessElasticConfiguration configuration, ILoggerFactory loggerFactory) : base(loggerFactory)
+    {
+        _configuration = configuration;
+        MigrationType = MigrationType.VersionedAndResumable;
+        Version = 10;
+    }
+
+    public override async Task RunAsync(MigrationContext context)
+    {
+        var response = await _configuration.Client.Indices.PutMappingAsync<PersistentEvent>(mapping => mapping
+            .Indices($"{_configuration.Events.Name}-v*-*")
+            .AllowNoIndices(true)
+            .IgnoreUnavailable(true)
+            .Properties(properties => properties.Text(ev => ev.Environment,
+                text => text.Analyzer(EventIndex.LOWER_KEYWORD_ANALYZER)
+                    .Fields(fields => fields.Keyword("keyword", keyword => keyword.Normalizer("lowercase"))))), context.CancellationToken);
+        _logger.LogRequest(response);
+        if (!response.IsValidResponse)
+        {
+            throw new InvalidOperationException("Unable to add the event environment mapping: " + response.DebugInformation);
+        }
+    }
+}

src/Exceptionless.Core/Models/Event.cs

@@ -11,6 +11,25 @@ namespace Exceptionless.Core.Models;
 [DebuggerDisplay("Type: {Type}, Date: {Date}, Message: {Message}, Value: {Value}, Count: {Count}")]
 public class Event : IData, IJsonOnDeserialized
 {
+    private string? _environment;
+
+    /// <summary>
+    /// The deployment environment, such as production, staging, or development.
+    /// Missing or invalid names remain unspecified. Machine and runtime information is stored separately in data.@environment.
+    /// </summary>
+    [StringLength(64)]
+    public string? Environment
+    {
+        get => _environment;
+        set
+        {
+            string? name = value?.Trim();
+            _environment = String.IsNullOrEmpty(name) || name.Length > 64 || name.Any(Char.IsControl)
+                ? null
+                : name;
+        }
+    }
+
     /// <summary>
     /// The event type (ie. error, log message, feature usage). Check <see cref="KnownTypes">Event.KnownTypes</see> for standard event types.
     /// Nullable in transit; the pipeline infers a default before save. Validated as required on repository save.
@@ -102,6 +121,11 @@ void IJsonOnDeserialized.OnDeserialized()
             Data ??= [];
             foreach (var kvp in ExtensionData)
             {
+                // Ingestion leaves deployment environments in extension data to retain the
+                // original key casing and value, including case-distinct legacy properties.
+                if (String.Equals(kvp.Key, nameof(Environment), StringComparison.OrdinalIgnoreCase))
+                    Environment = kvp.Value.ValueKind == JsonValueKind.String ? kvp.Value.GetString() : null;
+
                 object? value = JsonElementConverter.Convert(kvp.Value);
                 EventDataNormalizer.Set(Data, kvp.Key, value);
             }
@@ -112,7 +136,7 @@ void IJsonOnDeserialized.OnDeserialized()
 
     protected bool Equals(Event other)
     {
-        return String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data);
+        return String.Equals(Environment, other.Environment) && String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data);
     }
 
     public override bool Equals(object? obj)
@@ -138,6 +162,10 @@ public override int GetHashCode()
             hashCode = (hashCode * 397) ^ (Geo?.GetHashCode() ?? 0);
             hashCode = (hashCode * 397) ^ Value.GetHashCode();
             hashCode = (hashCode * 397) ^ (Data?.GetCollectionHashCode(_exclusions) ?? 0);
+            if (Environment is not null)
+            {
+                hashCode = (hashCode * 397) ^ Environment.GetHashCode();
+            }
             return hashCode;
         }
     }

src/Exceptionless.Core/Models/EventSummaryModel.cs

@@ -2,6 +2,7 @@
 
 public record EventSummaryModel : SummaryData
 {
+    public string? Environment { get; set; }
     public DateTimeOffset Date { get; set; }
     public string ProjectId { get; set; } = null!;
     public string? ProjectName { get; set; }

src/Exceptionless.Core/Models/Queues/EventPostInfo.cs

@@ -1,4 +1,6 @@
-namespace Exceptionless.Core.Queues.Models;
+using System.Text.Json.Serialization;
+
+namespace Exceptionless.Core.Queues.Models;
 
 public record EventPostInfo
 {
@@ -7,6 +9,10 @@ public record EventPostInfo
     public string? CharSet { get; init; }
     public string? MediaType { get; init; }
     public int ApiVersion { get; init; }
+    // Internal GET submissions and retries already contain normalized event data.
+    // Missing metadata on older queued posts retains the normal ingestion path.
+    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
+    public bool IsNormalized { get; init; }
     public string? UserAgent { get; init; }
     public string? ContentEncoding { get; init; }
     public string? IpAddress { get; init; }

src/Exceptionless.Core/Models/WebHookEvent.cs

@@ -14,6 +14,7 @@ public WebHookEvent(string baseUrl)
     public DateTimeOffset? OccurrenceDate { get; init; }
     public TagSet? Tags { get; init; }
     public string? Type { get; init; }
+    public string? Environment { get; init; }
     public string? Source { get; init; }
     public string? Message { get; init; }
     public string ProjectId { get; init; } = null!;

src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs

@@ -1,7 +1,9 @@
 using System.Text.Json;
+using System.Text.Json.Serialization.Metadata;
 using Exceptionless.Core.Extensions;
 using Exceptionless.Core.Models;
 using Exceptionless.Core.Pipeline;
+using Exceptionless.Core.Serialization;
 using Microsoft.Extensions.Logging;
 
 namespace Exceptionless.Core.Plugins.EventParser;
@@ -10,16 +12,34 @@ namespace Exceptionless.Core.Plugins.EventParser;
 public class JsonEventParserPlugin : PluginBase, IEventParserPlugin
 {
     private readonly JsonSerializerOptions _jsonOptions;
+    private readonly JsonSerializerOptions _normalizedJsonOptions;
 
     public JsonEventParserPlugin(AppOptions options, JsonSerializerOptions jsonOptions, ILoggerFactory loggerFactory) : base(options, loggerFactory)
     {
         // Create lenient parsing options — inbound events from older SDK clients may omit
         // non-nullable properties. We must not reject structurally valid events; the pipeline
         // handles missing/null values gracefully downstream.
-        _jsonOptions = new JsonSerializerOptions(jsonOptions) { RespectNullableAnnotations = false };
+        _normalizedJsonOptions = new JsonSerializerOptions(jsonOptions) { RespectNullableAnnotations = false };
+        _jsonOptions = new JsonSerializerOptions(_normalizedJsonOptions)
+        {
+            // Preserve the original root value once, at ingestion. Repeating this during
+            // storage or API deserialization would accumulate duplicate custom data.
+            TypeInfoResolver = (jsonOptions.TypeInfoResolver ?? new DefaultJsonTypeInfoResolver())
+                .WithAddedModifier(EventEnvironmentConverter.ConfigureIngestionProperty)
+        };
     }
 
     public List<PersistentEvent>? ParseEvents(string input, int apiVersion, string? userAgent)
+    {
+        return ParseEvents(input, apiVersion, _jsonOptions);
+    }
+
+    internal List<PersistentEvent>? ParseNormalizedEvents(string input)
+    {
+        return ParseEvents(input, 2, _normalizedJsonOptions);
+    }
+
+    private List<PersistentEvent>? ParseEvents(string input, int apiVersion, JsonSerializerOptions jsonOptions)
     {
         if (apiVersion < 2)
             return null;
@@ -31,7 +51,7 @@ public JsonEventParserPlugin(AppOptions options, JsonSerializerOptions jsonOptio
             {
                 try
                 {
-                    var ev = JsonSerializer.Deserialize<PersistentEvent>(input, _jsonOptions);
+                    var ev = JsonSerializer.Deserialize<PersistentEvent>(input, jsonOptions);
                     if (ev is not null)
                         events.Add(ev);
                 }
@@ -47,7 +67,7 @@ public JsonEventParserPlugin(AppOptions options, JsonSerializerOptions jsonOptio
             {
                 try
                 {
-                    var parsedEvents = JsonSerializer.Deserialize<PersistentEvent[]>(input, _jsonOptions);
+                    var parsedEvents = JsonSerializer.Deserialize<PersistentEvent[]>(input, jsonOptions);
                     if (parsedEvents is { Length: > 0 })
                         events.AddRange(parsedEvents.Where(e => e is not null));
                 }