Hi awesome person. In https://developer.apple.com/forums/thread/709170 I had the following code except jitted=True. I changed jitted to False. Running the following
elbo_loss_curve = tfp.vi.fit_surrogate_posterior(
target_log_prob_fn=model.joint_distribution(
observed_time_series=df_train["coverage"]).log_prob,
surrogate_posterior=variational_posteriors,
optimizer=tf.optimizers.Adam(learning_rate=0.1),
num_steps=num_variational_steps,
jit_compile=False)
Gives me the error below. I am not doing any with statements.
This is holding me back. Any chance I could get updates on this or the other ticket? Thanks. Joseph. For the other bits, this is real fast! thank you.
File "/Users/joseph/Downloads/structural_time_series_modeling_case_studies_atmospheric_co2_and_electricity_demand.py", line 104, in <module>
elbo_loss_curve = tfp.vi.fit_surrogate_posterior(
File "/opt/homebrew/Caskroom/miniforge/base/envs/tf/lib/python3.10/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/homebrew/Caskroom/miniforge/base/envs/tf/lib/python3.10/site-packages/tensorflow_probability/python/vi/optimization.py", line 751, in fit_surrogate_posterior
return tfp_math.minimize(complete_variational_loss_fn,
File "/opt/homebrew/Caskroom/miniforge/base/envs/tf/lib/python3.10/site-packages/tensorflow_probability/python/math/minimize.py", line 610, in minimize
_, traced_values = _minimize_common(
File "/opt/homebrew/Caskroom/miniforge/base/envs/tf/lib/python3.10/site-packages/tensorflow_probability/python/math/minimize.py", line 153, in _minimize_common
initial_optimizer_state) = optimizer_step_fn(
File "/opt/homebrew/Caskroom/miniforge/base/envs/tf/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/opt/homebrew/Caskroom/miniforge/base/envs/tf/lib/python3.10/site-packages/tensorflow/python/eager/execute.py", line 54, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation monte_carlo_variational_loss/build_factored_surrogate_posterior/Normal_trainable_variables/Identity_1/ReadVariableOp: Could not satisfy explicit device specification '' because the node {{colocation_node monte_carlo_variational_loss/build_factored_surrogate_posterior/Normal_trainable_variables/Identity_1/ReadVariableOp}} was colocated with a group of nodes that required incompatible device '/job:localhost/replica:0/task:0/device:GPU:0'. All available devices [/job:localhost/replica:0/task:0/device:CPU:0, /job:localhost/replica:0/task:0/device:GPU:0].
Colocation Debug Info:
Colocation group had the following types and supported devices:
Root Member(assigned_device_name_index_=2 requested_device_name_='/job:localhost/replica:0/task:0/device:GPU:0' assigned_device_name_='/job:localhost/replica:0/task:0/device:GPU:0' resource_device_name_='/job:localhost/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU] possible_devices_=[]
ReadVariableOp: GPU CPU
ResourceApplyAdam: CPU
_Arg: GPU CPU
Colocation members, user-requested devices, and framework assigned devices, if any:
monte_carlo_variational_loss_build_factored_surrogate_posterior_normal_trainable_variables_identity_1_readvariableop_resource (_Arg) framework assigned device=/job:localhost/replica:0/task:0/device:GPU:0
adam_adam_update_resourceapplyadam_m (_Arg) framework assigned device=/job:localhost/replica:0/task:0/device:GPU:0
adam_adam_update_resourceapplyadam_v (_Arg) framework assigned device=/job:localhost/replica:0/task:0/device:GPU:0
monte_carlo_variational_loss/build_factored_surrogate_posterior/Normal_trainable_variables/Identity_1/ReadVariableOp (ReadVariableOp)
monte_carlo_variational_loss/build_factored_surrogate_posterior_1/Normal_trainable_variables/Identity_1/ReadVariableOp (ReadVariableOp)
monte_carlo_variational_loss/build_factored_surrogate_posterior_2/Normal_trainable_variables/Identity_1/ReadVariableOp (ReadVariableOp)
Adam/Adam/update/ResourceApplyAdam (ResourceApplyAdam) /job:localhost/replica:0/task:0/device:GPU:0
Identity_9/ReadVariableOp (ReadVariableOp)
[[{{node monte_carlo_variational_loss/build_factored_surrogate_posterior/Normal_trainable_variables/Identity_1/ReadVariableOp}}]] [Op:__inference_optimizer_step_13601]
