Explorers
Explorers will help you to compare 1000s of AI experiments with a few clicks. Explorers are the main tools that Aim is built around.
In this section we will go through the Aim explorers, introduce their features and how to use them.
Metrics Explorer
Overview
Use Metrics explorer to search and compare 1000s of ML training metrics.
The Metrics Explorer allows you to search, group and compare your metrics. Due to this and number of other visual features on the Metrics Explorer, you will save considerable amounts of time when comparing experiments- compared to other open-source experiment tracking tools.
The Metric Explorer has the following main sections:
Metrics Select: to select the metrics for exploration
Search bar: to query the runs for exploration
Charts explorer: the space where the metrics are rendered
Metrics modifiers: all the grouping, chart division and other metrics modifier tools
Context table: the full information about the selected metrics is available
There is also an advanced mode of search is available too where you can use the full Aim QL (more on this further in this section).
There are two ways you can query metrics and runs
An overview of what you can do with queried metrics - the modifiers:
Select metrics and query runs
On the Metrics Explorer, there is + Metrics
button.
Once pressed, a dropdown will appear with all your tracked metrics and their contexts flattened.
The dropdown is searchable - so you can get to your metric of interest within a stroke!
The Search bar is located below the + Metric
button. It allows to do a pythonic query (that is eval-ed as python statement) over every param you have tracked.
Search runs with Aim QL
Advanced Search mode
Once you press the Enable advanced search mode
button underneath the main Search
button, it will enable the full Aim QL search editor - to query the metrics, the runs via full Aim QL
Here is an example:
((metric.name == 'bleu' and metric.context.subset == 'val') or (metric.name == 'loss' and metric.context.subset == 'train'))
and 1e-5 < run.hparams.learning_rate < 1e-2
Group by any parameter
Grouping selected metrics by any tracked params will allow you to quickly distinguish the most impactful params, decisions you have made (the preprocessing steps, the hyperparams etc).
The parameters include not only the ones you have tracked but also the native Aim objects too such as
metric.name
metric.context.[context_key]
run.hash
There are several ways you can group the selected metrics and runs - by color, by stroke and by chart.
Group by Color
Use this to divide the selected metrics into different clusters based on selected values of params. Each cluster gets colored differently.
There are a number of options available when grouping
group by values - divides into clusters as per the values of selected params)
reverse grouping - divides into clusters by every param except for the chosen one.
The grouping colors are picked randomly, however it is possible to fix with the advanced coloring features.
Here are the features in the advanced mode:
Fix the colors of the grouping
Control the color palette to use during the grouping
Group by Stroke
Groups the metrics by a stroke style. Has all the rest of the other features available on the color grouping except the advanced mode.
Group by Chart
The end result of using this feature: divides into subplots based on the value of the selected params. Why this is a grouping mechanism? It groups the metrics belonging to the same group into separate charts.
Aggregate metrics
The metrics aggregation helps to quickly see the trends of each individual group of metrics. See more about metrics grouping.
There are two aspects of aggregation you can control:
the trend-line
the area that the group of metricsc take
The trend-line calculation methods:
Mean
Median
Min
Max
The area calcualtion methods:
None (when you’d like to remove the area)
Min/Max
Mean +/- Standard Deviation
Mean +/- Standard Error
Confidence Interval (95%)
Pls see the screenshot:
Axes properties
Axes properties section is for aligning metrics by time, epoch or another metric and for setting axes range manually.
Alignment:
Following types of metrics alignment are available: Step, Epoch, Relative Time, Absolute Time and Custom Metric. By default, metrics aligned by Step.
Step
By setting metrics alignment to Step, x-axis values will represent the steps of tracked metrics.
Epoch
By setting metrics alignment to Epoch, x-axis values will represent the epochs of tracked metrics.
Relative Time
By setting metrics alignment to Relative Time, x-axis values will represent by HH:mm:ss
, duration of tracking process.
Absolute Time
By setting metrics alignment to Absolute Time, x-axis values will represent by date HH:mm:ss D MMM, YY
, since the start date of the first run until the last run.
Custom Metric
By setting metrics alignment to Custom Metric, x-axis values will represent selected metric values, you can detect correlations between queried metrics and selected metric.
Set axes range:
To fix an axis range across all the charts, set the corresponding axis minimum and maximum bounds in the form.
Axes Scale
Axes Scale section gives ability to display axes scale’s linear or logarithmic.
By default, axes scale’s are Linear.
Linear Scale
X-axis scale: Linear, Y-axis scale: Log
X-axis scale: Log, Y-axis scale: Linear
Log Scale
Chart Smoothing
While smoothing the chart, the data points are modified so individual points higher than the adjacent points (presumably because of noise) are reduced, and points that are lower than the adjacent points are increased leading to a smoother signal. You can select curve interpolation methods: Linear or Cubic. By default, chart smoothing in Exponential moving average mode and curve interpolation method is Linear.
Exponential moving average
An exponential moving average, also known as an exponentially weighted moving average (EWMA), is a first-order infinite impulse response filter that applies weighting factors which decrease exponentially.
Centered moving average
When you center the moving averages, the data points are placed at the center of the range rather than the end of it. This is done to position the moving average values at their central positions in time.
Ignore outliers
An outlier is an observation that lies an abnormal distance from other values in a random sample from a population. Examination of the data for unusual observations that are far removed from the mass of data. These points are often referred to as outliers.
Excluding outliers can cause your results to become statistically significant. By default, outliers are ignored.
Highlight Modes
Highlighting functionality is useful for filtering metrics and highlight only hovered metric. Following types of highlighting mode are available: Highlight Off, Highlight Metric on Hover, and Highlight Run on Hover. By default, highlighting mode is the Highlight Run on Hover.
Highlight Off
By setting Highlight mode Off, there is no highlighting functionality on hover.
Highlight Metric on Hover
By setting Highlight mode Metric on Hover, mouse point closest metric highlights and other metrics displays with opacity.
Highlight Run on Hover
By setting Highlight mode Run on Hover, mouse point closest metric highlights and highlighted metric corresponding run also highlights other metrics displays with opacity.
Set tooltip parameters
You can select tooltip parameters to show params and values in tooltip Params section. You can select hide or show button to display or hide tooltip on hover.
Apply zoom on charts
Zoom In
Zoom Out
Export chart as image
Metric explorer also, gives ability to export your chart as image.
By clicking export button
from control panel, will be opened chart preview modal.
You can change exportable chart image width
, single chart height
, set image name
and format
.
Following formats of chart export are available: SVG
, JPEG
, PNG
.
Images Explorer
Overview
Track intermediate images search easily by using select form functional compare them on the Images Explorer by using reach controls panel.
Features:
Query any image
Use select form to easily query any image. There are two option to query images using dropdown, by using Aim QL language and advance mode for Aim QL.
Click on Images button
Select options you are want to use in query
Click on the Search button
Click on pencel icon in the right side of select form to show input
Type advance Aim QL query
Click on the Search button
Group image by run parameters
Use select grouping dropdown which is located in the right top corner of the image explore page.
Click on grouping button
Select fields by which you want to groupe images
Grouping will be apply after each field selection also you can select grouping mode (Group or Reverse)
Image explorer right controls panel
Any change in controls will help to explore images better on the workspace
Images size manipulation control
Click on image property button
Select value from dropdown to align image. (by default dropdown value is
Height
). Use slider to configure value for scale relative to window size by default scale value is15%
.By height
By width
Original size
Use image rendering variation by default value of this control is
Pixelated
Images sorting control
Click on image sorting button
Select fields for sorting images. Selection ordering is meaningful and data will be sorting by selection order. Bellow is visible Ordered By list where contains all selected fields from dropdown. You can remove any already selected field by clicking on
x
icon or change sorting direction by clicking radio button Asc or Desc. Default selected direction is Asc.For reset all existing sorting fields you can simply click on Reset Sorting button
Set tooltip parameters
You can select tooltip parameters to show params and values in tooltip Params section. You can select hide or show button to display or hide tooltip on hover.
Params Explorer
Overview
Params explorer helps you to represent high dimensional data as a multi-dimensional visualization. Features:
Query any metrics and params
Select params and metrics from dropdown
Search runs with Aim QL
Grouping
Group by color, stroke, or chart with selected parameters
Curve interpolation
By clicking on the Curve interpolation button in the Controls panel, it’s possible to make lines from straight to curve to show correlations between non-adjacent axes.
Color indicator
By clicking on the Color indicator button in the Controls panel, it’s possible to turn on lines gradient coloring by the last dimension.
Scatters Explorer
Scatter explorer gives ability to visualize correlations between metric last value data with hyper-parameter.
It represents graph where corresponding values from a set of data are placed as points on a coordinate plane. A relationship between the points is sometimes shown to be positive, negative, strong, or weak.
Abilities provided by Scatter explorer
Select params and metrics from X
and Y
axes dropdowns to align metric last value data with hyper-parameter.
X axis
Y axis
Also, you can search runs with Aim QL
Easily group data by color and chart with selected parameters.
By
Color
By
Chart
A trend line is a straight line that best represents the points on a scatter plot
. The trend line may go through some points but need not go through them all.
From trend line options popover you can change regression from Linear
(by default) to LOESS
(locally weighted smoothing), which creates a smooth line through a scatter plot
to help you to see relationship between variables and foresee trends. Also, you can change the bandwidth
with slider
Scatter explorer also, gives ability to export
your chart as image
.
By clicking export button
from control panel, will be opened chart preview modal.
You can change exportable chart image width
, single chart height
, set image name
and format
.
Following image formats are available export: SVG
, JPEG
, PNG
.