Make get_post_fcov methods return posterior covariance matrices of Gaussian Process

[shibukawar]

Hi PHYSBO developers! Thank you for developing and maintaining this amazing software.
I am enjoying the usability and efficiency of PHYSBO.
In this pull request, I want to make a tiny contribution.

Description

Obtain posterior covariance matrices of Gaussian Process

Changes

Propagate the diag flag from get_post_fcov methods

Why is this necessary?

We need a posterior covariance matrix for novel algorithms.

Demo

Click here to expand...

import numpy as np
import scipy
import physbo
import itertools

window_num=10001
x_max = 2.0
x_min = -2.0

X = np.linspace(x_min,x_max,window_num).reshape(window_num, 1)

class simulator:

    def __call__(self, action):
        action_idx = action[0]
        x = X[action_idx][0]
        fx = 3.0*x**4 + 4.0*x**3 + 1.0
        fx_list.append(fx)
        x_list.append(X[action_idx][0])

        print ("*********************")
        print ("Present optimum interactions")

        print ("x_opt=", x_list[np.argmin(np.array(fx_list))])

        return -fx

policy = physbo.search.discrete.policy(test_X=X)

policy.set_seed(0)

fx_list=[]
x_list = []
res = policy.random_search(
	max_num_probes=20,
	simulator=simulator()
)

res = policy.bayes_search(
	max_num_probes=50,
	simulator=simulator(),
	score='TS',
	interval=0,
	num_rand_basis=500
)

print(policy.get_post_fcov(X, diag=True))
# -> [0.93196067 0.92704693 0.9221549  ... 6.8052756  6.82997947 6.85475494]

print(policy.get_post_fcov(X, diag=False)) # Posterior covariance matrix is returned.
# -> [[0.93196067 0.92949971 0.92704144 ... 0.27751007 0.27805598 0.27860213]
 [0.92949971 0.92704693 0.92459684 ... 0.27633487 0.27687851 0.27742238]
 [0.92704144 0.92459684 0.9221549  ... 0.27516163 0.275703   0.27624459]
 ...
 [0.27751007 0.27633487 0.27516163 ... 6.8052756  6.8176145  6.82996308]
 [0.27805598 0.27687851 0.275703   ... 6.8176145  6.82997947 6.84235414]
 [0.27860213 0.27742238 0.27624459 ... 6.82996308 6.84235414 6.85475494]]