The calculate_metrics
function computes several evaluation metrics to assess the performance of the model in predicting temperature and humidity. Here's a detailed description of each metric:
Accuracy Metrics
def calculate_metrics(self, y_true, y_pred):
metrics = {}
# Initialize sum values for all metrics
r2_temperature_sum = 0
r2_humidity_sum = 0
rrmse_temperature_sum = 0
rrmse_humidity_sum = 0
acc_sum = 0
err_sum = 0
temp_acc_sum = 0
humidity_acc_sum = 0
# Loop through each time step to calculate metrics
for t in range(y_true.shape[1]):
# R2 and RRMSE calculations
r2_temperature_sum += r2_score(y_true[:, t, 0], y_pred[:, t, 0])
r2_humidity_sum += r2_score(y_true[:, t, 1], y_pred[:, t, 1])
rrmse_temperature_sum += self.calculate_rrmse(
y_true[:, t, 0], y_pred[:, t, 0]
)
rrmse_humidity_sum += self.calculate_rrmse(
y_true[:, t, 1], y_pred[:, t, 1])
# Accuracy calculations
acc, err, temp_acc, humidity_acc = self.calculate_accuracy(
y_true[:, t, :], y_pred[:, t, :]
)
acc_sum += acc
err_sum += err
temp_acc_sum += temp_acc
humidity_acc_sum += humidity_acc
# Average metrics over all time steps
metrics["r2_temperature"] = r2_temperature_sum / y_true.shape[1]
metrics["r2_humidity"] = r2_humidity_sum / y_true.shape[1]
metrics["r2_average"] = (
metrics["r2_temperature"] + metrics["r2_humidity"]) / 2
metrics["rrmse_temperature"] = rrmse_temperature_sum / y_true.shape[1]
metrics["rrmse_humidity"] = rrmse_humidity_sum / y_true.shape[1]
metrics["rrmse_average"] = (
metrics["rrmse_temperature"] + metrics["rrmse_humidity"]
) / 2
# Average accuracy metrics
metrics["acc"] = acc_sum / y_true.shape[1]
metrics["err"] = err_sum / y_true.shape[1]
metrics["temp_acc"] = temp_acc_sum / y_true.shape[1]
metrics["humidity_acc"] = humidity_acc_sum / y_true.shape[1]
print("Metrics: ", metrics)
return metrics
Rsquared (R2)
r2_temperature
andr2_humidity
are calculated separately for temperature and humidity predictions using ther2_score
function. These are summed for each time step and then averaged across all time steps to get
metrics["r2_temperature"]
andmetrics["r2_humidity"]
. metrics["r2_average"]
is the average ofmetrics["r2_temperature"]
andmetrics["r2_humidity"]
, representing the overall R2 score for both temperature and humidity predictions.
Relative Root Mean Square Error (RRMSE)
 This is a normalized measure of the differences between values predicted by a model and the values actually observed from the environment that is being modeled.
 Like R2, RRMSE is also calculated separately for temperature and humidity (
rrmse_temperature
andrrmse_humidity
), averaged over all time steps. metrics["rrmse_average"]
is the average ofmetrics["rrmse_temperature"]
andmetrics["rrmse_humidity"]
, indicating the overall RRMSE for both predictions.
def calculate_rrmse(self, y_true, y_pred):
return np.sqrt(np.mean((y_true  y_pred) ** 2) / np.mean(y_true**2))
Accuracy (ACC)
The calculate_accuracy
function is designed to evaluate the prediction accuracy with respect to predefined thresholds for temperature and humidity:
def calculate_accuracy(
self, y_true, y_pred, temp_threshold=1, humidity_threshold=10
):
temp_error = np.abs(y_pred[:, 0]  y_true[:, 0])
humidity_error = np.abs(y_pred[:, 1]  y_true[:, 1])
temp_acc = np.mean(temp_error < temp_threshold)
humidity_acc = np.mean(humidity_error < humidity_threshold)
acc = (temp_acc + humidity_acc) / 2
err = 1  acc
return acc, err, temp_acc, humidity_acc

Temperature Accuracy (
temp_acc
): Calculated by the absolute error between the predicted and actual temperatures.
 A prediction is considered accurate if the absolute error is less than the
temp_threshold
. temp_acc
is the proportion of temperature predictions within the acceptable error threshold.

Humidity Accuracy (
humidity_acc
): Calculated by the absolute error between the predicted and actual humidity levels.
 A prediction is considered accurate if the absolute error is less than the
humidity_threshold
. humidity_acc
is the proportion of humidity predictions within the acceptable error threshold.

Overall Accuracy (
acc
): This is the average of
temp_acc
andhumidity_acc
, representing the proportion of all predictions that are within their respective acceptable error thresholds.
 This is the average of

Error Rate (
err
): Calculated as one minus the overall accuracy.
 Represents the proportion of predictions that fall outside the acceptable error thresholds.
These accuracy metrics provide a comprehensive view of the model's performance, taking into account the accuracy of both temperature and humidity predictions. The use of error thresholds offers the flexibility to define what is considered an acceptable prediction, which can be tailored to the specific needs of the application.