Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix MiqRequestTask ResourceAction with a Workflow #23123

Merged

Conversation

agrare
Copy link
Member

@agrare agrare commented Aug 2, 2024

If a ServiceTemplate has a ResourceAction with a
configuration_script_payload as its entrypoint this was not being run by MiqRequestTask#deliver_queue.

Introduced by #23039

We did have a spec test covering this case but it was changed by https://github.com/ManageIQ/manageiq/pull/23039/files#diff-169a716c426d776976d164d5088f0f2dc18360c2417272e40af19f153e86f437 so the regression wasn't caught.

Service Templates

Pure workflows based service catalog item

#<ServiceTemplate:0x00007f76a8fbb1b8
 id: 1,
 name: "Provision a VM with Workflows",
 description: nil,
 guid: "88bab357-d1e1-4f3c-a4d1-e586bd28be72",
 type: nil,
 service_template_id: nil,
 options: {},
 created_at: Tue, 13 Aug 2024 18:54:41.327407000 UTC  00:00,
 updated_at: Tue, 13 Aug 2024 18:54:41.327407000 UTC  00:00,
 display: true,
 evm_owner_id: nil,
 miq_group_id: 2,
 service_type: "atomic",
 prov_type: "generic",
 provision_cost: nil,
 service_template_catalog_id: 1,
 long_description: nil,
 tenant_id: 1,
 generic_subtype: "custom",
 deleted_on: nil,
 internal: false,
 zone_id: 2,
 currency_id: nil,
 price: nil>
>> ServiceTemplate.first.resource_actions
=> 
[#<ResourceAction:0x00007f76a93dec18
  id: 6,
  action: "Provision",
  dialog_id: 2,
  resource_id: 1,
  resource_type: "ServiceTemplate",
  created_at: Tue, 13 Aug 2024 18:54:41.335580000 UTC  00:00,
  updated_at: Tue, 13 Aug 2024 18:54:41.335580000 UTC  00:00,
  ae_namespace: nil,
  ae_class: nil,
  ae_instance: nil,
  ae_message: nil,
  ae_attributes: {:service_action=>"Provision"},
  configuration_template_id: nil,
  configuration_template_type: nil,
  configuration_script_id: 10>,
 #<ResourceAction:0x00007f76a93dea10
  id: 7,
  action: "Retirement",
  dialog_id: 2,
  resource_id: 1,
  resource_type: "ServiceTemplate",
  created_at: Tue, 13 Aug 2024 18:54:41.340149500 UTC  00:00,
  updated_at: Tue, 13 Aug 2024 18:54:41.340149500 UTC  00:00,
  ae_namespace: "Service/Retirement/StateMachines",
  ae_class: "ServiceRetirement",
  ae_instance: "Default",
  ae_message: nil,
  ae_attributes: {:service_action=>"Retirement"},
  configuration_template_id: nil,
  configuration_template_type: nil,
  configuration_script_id: nil>]

VMware provision with workflows built-in

#<ServiceTemplate:0x00007f76a92bdc08
 id: 2,
 name: "Provision VMware",
 description: nil,
 guid: "4d3b3854-d7df-49ca-9fc7-ed44749dbbca",
 type: nil,
 service_template_id: nil,
 options: {},
 created_at: Tue, 13 Aug 2024 18:58:39.108522000 UTC  00:00,
 updated_at: Tue, 13 Aug 2024 18:58:39.108522000 UTC  00:00,
 display: true,
 evm_owner_id: nil,
 miq_group_id: 2,
 service_type: "atomic",
 prov_type: "vmware",
 provision_cost: nil,
 service_template_catalog_id: 1,
 long_description: nil,
 tenant_id: 1,
 generic_subtype: nil,
 deleted_on: nil,
 internal: false,
 zone_id: 2,
 currency_id: nil,
 price: nil>
>> ServiceTemplate.last.resource_actions
[#<ResourceAction:0x00007f76a91d8b80
  id: 8,
  action: "Provision",
  dialog_id: 1,
  resource_id: 2,
  resource_type: "ServiceTemplate",
  created_at: Tue, 13 Aug 2024 18:58:39.116500000 UTC  00:00,
  updated_at: Tue, 13 Aug 2024 18:58:39.116500000 UTC  00:00,
  ae_namespace: nil,
  ae_class: nil,
  ae_instance: nil,
  ae_message: nil,
  ae_attributes: {:service_action=>"Provision"},
  configuration_template_id: nil,
  configuration_template_type: nil,
  configuration_script_id: 3>,
 #<ResourceAction:0x00007f76a91d88b0
  id: 9,
  action: "Retirement",
  dialog_id: 1,
  resource_id: 2,
  resource_type: "ServiceTemplate",
  created_at: Tue, 13 Aug 2024 18:58:39.121614000 UTC  00:00,
  updated_at: Tue, 13 Aug 2024 18:58:39.121614000 UTC  00:00,
  ae_namespace: "Service/Retirement/StateMachines",
  ae_class: "ServiceRetirement",
  ae_instance: "Default",
  ae_message: nil,
  ae_attributes: {:service_action=>"Retirement"},
  configuration_template_id: nil,
  configuration_template_type: nil,
  configuration_script_id: nil>]

Ordering

Pure Workflows

You get a single MiqRequestTask that is a ServiceTemplateProvisionTask

#<ServiceTemplateProvisionTask:0x00007f76acbceee8
 id: 13,
 description: "Provisioning [Provision a VM with Workflows] for Service [Provision a VM with Workflows]",
 state: "provisioned",
 request_type: "clone_to_service",
 userid: "admin",
 options:
  {:dialog=>{"dialog_vm_name"=>"ag-prov-test", "dialog_provider"=>"4", "dialog_source_template"=>"vm-63", "dialog_verify_ssl"=>"[FILTERED]"},
   :workflow_settings=>{:resource_action_id=>6, :dialog_id=>2},
   :initiator=>nil,
   :src_id=>1,
   :request_options=>{:submit_workflow=>true, :init_defaults=>false},
   :cart_state=>"ordered",
   :requester_group=>"EvmGroup-super_administrator",
   :executed_on_servers=>[1],
   :delivered_on=>2024-08-14 16:19:07.823234011 UTC,
   :pass=>0},
 created_on: Wed, 14 Aug 2024 16:19:07.878478000 UTC  00:00,
 updated_on: Wed, 14 Aug 2024 16:25:27.275847000 UTC  00:00,
 message: "Service does not have children processes",
 status: "Ok",
 type: "ServiceTemplateProvisionTask",
 miq_request_id: 8,
 source_id: 1,
 source_type: "ServiceTemplate",
 destination_id: 7,
 destination_type: "Service",
 miq_request_task_id: nil,
 phase: nil,
 phase_context: {},
 tenant_id: 1,
 cancelation_status: nil,
 conversion_host_id: nil>
>> MiqRequestTask.first.resource_action
=> 
#<ResourceAction:0x00007f76aabe1a40
 id: 6,
 action: "Provision",
 dialog_id: 2,
 resource_id: 1,
 resource_type: "ServiceTemplate",
 created_at: Tue, 13 Aug 2024 18:54:41.335580000 UTC  00:00,
 updated_at: Tue, 13 Aug 2024 18:54:41.335580000 UTC  00:00,
 ae_namespace: nil,
 ae_class: nil,
 ae_instance: nil,
 ae_message: nil,
 ae_attributes: {:service_action=>"Provision"},
 configuration_template_id: nil,
 configuration_template_type: nil,
 configuration_script_id: 10>

This was where we had the original https://github.com/ManageIQ/manageiq/pull/22511/files#diff-8dfa5b51b60d86d6591a7daba20e32a847ffd140f057a52b251dccb5b82984faR142-R143 code to execute this.

deliver_queue here is called by:

 "/home/grare/adam/src/manageiq/manageiq/app/models/miq_request_task.rb:146:in `deliver_queue'",
 "/home/grare/adam/src/manageiq/manageiq/app/models/miq_request.rb:472:in `block in create_request_tasks'",
 "/home/grare/adam/src/manageiq/manageiq/app/models/miq_request.rb:469:in `each'",
 "/home/grare/adam/src/manageiq/manageiq/app/models/miq_request.rb:469:in `create_request_tasks'"

VMware MiqProvision with Workflows

With this you get 3 MiqRequestTasks, 2 ServiceTemplateProvisionTask and 1 ManageIQ::Providers::Vmware::InfraManager::Provision / MiqProvision task.

Top level service template provision task:

 #<ServiceTemplateProvisionTask:0x00007fb0650d8ca8
  id: 15,
  description: "Provisioning [Provision VMware] for Service [Provision VMware]",
  state: "pending",
  request_type: "clone_to_service",
  userid: "admin",
  options:
   {:dialog=>{"dialog_text_box_1"=>"Press submit to submit"},
    :workflow_settings=>{:resource_action_id=>8, :dialog_id=>1},
    :initiator=>nil,
    :src_id=>2,
    :request_options=>{:submit_workflow=>true, :init_defaults=>false},
    :cart_state=>"ordered",
    :requester_group=>"EvmGroup-super_administrator",
    :executed_on_servers=>[1],
    :delivered_on=>2024-08-14 16:33:28.73655658 UTC,
    :pass=>0},
  created_on: Wed, 14 Aug 2024 16:33:28.796296000 UTC  00:00,
  updated_on: Wed, 14 Aug 2024 16:33:30.732604000 UTC  00:00,
  message: "Service_Template_Provisioning - Request Created",
  status: "Ok",
  type: "ServiceTemplateProvisionTask",
  miq_request_id: 10,
  source_id: 2,
  source_type: "ServiceTemplate",
  destination_id: 9,
  destination_type: "Service",
  miq_request_task_id: nil,
  phase: nil,
  phase_context: {},
  tenant_id: 1,
  cancelation_status: nil,
  conversion_host_id: nil>

"intermediate" service tempalate provision task:

 #<ServiceTemplateProvisionTask:0x00007fb0650d7a38
  id: 16,
  description: "Provisioning VM [DC0_H0_VM0] for Service [DC0_H0_VM0]",
  state: "pending",
  request_type: "clone_to_service",
  userid: "admin",
  options:
   {:dialog=>{},
    :workflow_settings=>{:resource_action_id=>8, :dialog_id=>1},
    :initiator=>nil,
    :src_id=>1,
    :request_options=>{:submit_workflow=>true, :init_defaults=>false},
    :cart_state=>"ordered",
    :requester_group=>"EvmGroup-super_administrator",
    :executed_on_servers=>[1],
    :delivered_on=>2024-08-14 16:33:28.73655658 UTC,
    :pass=>0,
    :scaling_idx=>1,
    :scaling_min=>1,
    :service_resource_id=>1,
    :parent_service_id=>9,
    :parent_task_id=>15},
  created_on: Wed, 14 Aug 2024 16:33:29.122321000 UTC  00:00,
  updated_on: Wed, 14 Aug 2024 16:33:30.728597000 UTC  00:00,
  message: nil,
  status: "Ok",
  type: "ServiceTemplateProvisionTask",
  miq_request_id: 10,
  source_id: 1,
  source_type: "MiqRequest",
  destination_id: 9,
  destination_type: "Service",
  miq_request_task_id: 15,
  phase: nil,
  phase_context: {},
  tenant_id: 1,
  cancelation_status: nil,
  conversion_host_id: nil>

And finally the MiqProvision task:

#<ManageIQ::Providers::Vmware::InfraManager::Provision:0x00007fb0650ce050
  id: 17,
  description: "Provision from [DC0_H0_VM0] to [ag-prov-test0004]",
  state: "pending",
  request_type: "template",
  userid: "admin",
  options:
   {:initial_pass=>true,
    :service_template_request=>false,
    :miq_request_dialog_name=>"miq_provision_vmware_dialogs_template",
    :requester_enabled=>["disabled"],
    :purpose_enabled=>["disabled"],
    :current_tab_key=>:network,
    :owner_phone=>nil,
    :owner_country=>nil,
    :owner_phone_mobile=>nil,
    :owner_title=>nil,
    :owner_first_name=>nil,
    :owner_manager=>nil,
    :owner_address=>nil,
    :owner_company=>nil,
    :owner_last_name=>nil,
    :owner_manager_mail=>nil,
    :owner_city=>nil,
    :owner_department=>nil,
    :owner_load_ldap=>nil,
    :owner_manager_phone=>nil,
    :owner_state=>nil,
    :owner_office=>nil,
    :owner_zip=>nil,
    :owner_email=>nil,
    :request_notes=>nil,
    :vm_tags=>[],
    :customization_template_script=>nil,
    :dns_servers=>nil,
    :sysprep_organization=>nil,
    :sysprep_password=>"[FILTERED]",
    :sysprep_server_license_mode=>["perServer", "Per server"],
    :ldap_ous=>[nil, nil],
    :sysprep_timezone=>[nil, nil],
    :dns_suffixes=>nil,
    :sysprep_product_id=>nil,
    :sysprep_identification=>["domain", "Domain"],
    :sysprep_per_server_max_connections=>"5",
    :sysprep_computer_name=>nil,
    :sysprep_workgroup_name=>"WORKGROUP",
    :sysprep_spec_override=>[false, 0],
    :addr_mode=>["dhcp", "DHCP"],
    :linux_host_name=>nil,
    :sysprep_domain_admin=>nil,
    :sysprep_change_sid=>[true, 1],
    :sysprep_domain_name=>[nil, nil],
    :sysprep_upload_file=>nil,
    :gateway=>nil,
    :ip_addr=>nil,
    :linux_domain_name=>nil,
    :sysprep_domain_password=>"[FILTERED]",
    :sysprep_auto_logon=>[true, 1],
    :sysprep_enabled=>["disabled", "<None>"],
    :sysprep_delete_accounts=>[false, 0],
    :sysprep_upload_text=>nil,
    :wins_servers=>nil,
    :subnet_mask=>nil,
    :sysprep_full_name=>nil,
    :sysprep_auto_logon_count=>[1, "1"],
    :root_password=>"[FILTERED]",
    :hostname=>nil,
    :placement_cluster_name=>[1, "DC0_H0"],
    :cluster_filter=>[nil, nil],
    :host_filter=>[nil, nil],
    :ds_filter=>[nil, nil],
    :rp_filter=>[nil, nil],
    :placement_auto=>[false, 0],
    :placement_folder_name=>[nil, nil],
    :placement_rp_name=>[1, "Default for Cluster DC0_H0"],
    :placement_dc_name=>[2, "DC0"],
    :number_of_vms=>[1, "1"],
    :vm_description=>nil,
    :vm_prefix=>nil,
    :vm_name=>"ag-prov-test",
    :pxe_server_id=>[nil, nil],
    :host_name=>nil,
    :provision_type=>["vmware", "VMware"],
    :linked_clone=>[nil, nil],
    :snapshot=>[nil, nil],
    :vm_filter=>[nil, nil],
    :schedule_type=>["immediately", "Immediately on Approval"],
    :vm_auto_start=>[true, 1],
    :schedule_time=>Wed, 14 Aug 2024 00:00:00.000000000 UTC  00:00,
    :retirement=>[0, "Indefinite"],
    :retirement_warn=>[604800, "1 Week"],
    :vlan=>["VM Network", "VM Network"],
    :mac_address=>nil,
    :disk_format=>["unchanged", "Default"],
    :allocated_disk_storage=>"10",
    :cpu_limit=>nil,
    :memory_limit=>nil,
    :number_of_sockets=>[1, "1"],
    :cores_per_socket=>[1, "1"],
    :cpu_reserve=>nil,
    :vm_memory=>["1024", "1024"],
    :memory_reserve=>nil,
    :cpu_hot_add=>[false, 0],
    :cpu_hot_remove=>[false, 0],
    :memory_hot_add=>[false, 0],
    :network_adapters=>[1, "1"],
    :start_date=>"8/14/2024",
    :start_hour=>"00",
    :start_min=>"00",
    :src_vm_id=>[1, "DC0_H0_VM0"],
    :name=>"Provision VMware",
    :description=>nil,
    :long_description=>nil,
    :provision_cost=>nil,
    :display=>true,
    :catalog_id=>"1",
    :dialog_id=>"1",
    :st_prov_type=>"vmware",
    :tenant_ids=>[],
    :zone_id=>"2",
    :currency=>nil,
    :code_currency=>"Price / Month",
    :price=>nil,
    :provision_configuration_script_id=>3,
    :provision_entry_point_type=>"embedded_workflow",
    :fqname=>"provision.asl",
    :fqname_previous=>{:embedded_automate=>"/Service/Provisioning/StateMachines/ServiceProvision_Template/CatalogItemInitialization", :embedded_workflow=>nil},
    :reconfigure_configuration_script_id=>nil,
    :reconfigure_entry_point_type=>"embedded_automate",
    :reconfigure_fqname=>nil,
    :reconfigure_fqname_previous=>{:embedded_automate=>nil, :embedded_workflow=>nil},
    :retire_configuration_script_id=>nil,
    :retire_entry_point_type=>"embedded_automate",
    :retire_fqname=>"/Service/Retirement/StateMachines/ServiceRetirement/Default",
    :retire_fqname_previous=>{:embedded_automate=>"/Service/Retirement/StateMachines/ServiceRetirement/Default", :embedded_workflow=>nil},
    :namespace_configuration_script_id=>nil,
    :namespace_entry_point_type=>"embedded_automate",
    :namespace=>nil,
    :namespace_previous=>{:embedded_automate=>nil, :embedded_workflow=>nil},
    :available_dialogs=>{1=>"Submit", 2=>"Provision a VM with Workflows"},
    :service_type=>"atomic",
    "fqname_workflow"=>"provision.asl",
    :selected=>"cfp-3",
    "provision"=>"cfp-3",
    :placement_host_name=>[1, "localhost.localdomain"],
    :placement_ds_name=>[1, "LocalDS_0"],
    :src_vm_nics=>["ethernet-0"],
    :src_vm_lans=>["DC0_DVPG0"],
    :customize_enabled=>["disabled"],
    :src_ems_id=>[4, "vcsim"],
    :placement_storage_profile=>[nil, nil],
    :requester_group=>"EvmGroup-super_administrator",
    :pass=>0,
    :miq_force_unique_name=>[true, 1],
    :service_guid=>"1e0eb4b0-89fc-45ba-be96-cf7d1c528e3b",
    :service_resource_id=>1,
    :configuration_script_payload_id=>3,
    :owner_group=>"EvmGroup-super_administrator",
    :dns_domain=>nil,
    :vm_target_name=>"ag-prov-test0004",
    :vm_target_hostname=>"ag-prov-test000"},
  created_on: Wed, 14 Aug 2024 16:33:29.756507000 UTC  00:00,
  updated_on: Wed, 14 Aug 2024 16:33:30.720738000 UTC  00:00,
  message: "VM Provisioning - Request Created",
  status: "Ok",
  type: "ManageIQ::Providers::Vmware::InfraManager::Provision",
  miq_request_id: 10,
  source_id: 1,
  source_type: "VmOrTemplate",
  destination_id: nil,
  destination_type: nil,
  miq_request_task_id: 16,
  phase: nil,
  phase_context: {},
  tenant_id: 1,
  cancelation_status: nil,
  conversion_host_id: nil>

The problem

The issue here is that for a pure-workflows ServiceTemplateProvisionTask we have the workflow id in the resource_action, and we want to execute it for that ServiceTemplateProvisionTask

However, with a VMware based provision using a workflow we also have a ServiceTemplateProvisionTask that has a resource_action with the workflow id, BUT we don't want to execute that with workflows we only want to execute the MiqProvision task with workflows.

I tried moving the workflow definition down to MiqProvision to check options and leaving the ServiceTemplateProvisionTask to check the resource_action, but the issue is both have a valid resource_action workflow_id and we only want to call workflows on ONE of these types.

I think if we could get the pure-workflow ServiceTemplateProvisionTask to have the configuration_script_payload_id in options to indicate that we want to execute the workflow on this miq_request_task that would work.

@agrare agrare added the bug label Aug 2, 2024
@agrare agrare changed the title [WIP] Fix MiqRequestTask ResourceAction with a Workflow Fix MiqRequestTask ResourceAction with a Workflow Aug 2, 2024
@agrare agrare changed the title Fix MiqRequestTask ResourceAction with a Workflow [WIP] Fix MiqRequestTask ResourceAction with a Workflow Aug 2, 2024
@miq-bot miq-bot added the wip label Aug 2, 2024
@@ -140,12 140,13 @@ def deliver_queue(req_type = request_type, zone = nil)

_log.info("Queuing #{request_class::TASK_DESCRIPTION}: [#{description}]...")

workflow = ConfigurationScriptPayload.find(options[:configuration_script_payload_id]) if options[:configuration_script_payload_id]
workflow_id = resource_action&.configuration_script_id || options[:configuration_script_payload_id]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little complicated and I need to do some investigation to see if we can clean this up.

Basically if a Generic ServiceTemplate is created with an Embedded Workflow for an Entry Point that is set in the resource_action.
If a e.g. VMware ServiceTemplate is created the workflow goes to options[:configuration_script_payload_id] also when the child tasks are created.

@agrare agrare force-pushed the fix_miq_request_task_resource_action_workflow branch 2 times, most recently from 68efbb4 to b9c3dd3 Compare August 13, 2024 18:04
@agrare agrare changed the title [WIP] Fix MiqRequestTask ResourceAction with a Workflow Fix MiqRequestTask ResourceAction with a Workflow Aug 13, 2024
@miq-bot miq-bot removed the wip label Aug 13, 2024
@agrare agrare changed the title Fix MiqRequestTask ResourceAction with a Workflow [WIP] Fix MiqRequestTask ResourceAction with a Workflow Aug 13, 2024
@miq-bot miq-bot added the wip label Aug 13, 2024
Comment on lines 148 to 149
options[:configuration_script_payload_id] = workflow.id
options[:configuration_script_payload_id] = workflow_id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think leave this as it was

@kbrock
Copy link
Member

kbrock commented Aug 21, 2024

Is there a way to test that it executes this properly?
Not sure how to do this, so the current test may be the best option

If a ServiceTemplate has a ResourceAction with a
configuration_script_payload as its entrypoint this was not being run by
MiqRequestTask#deliver_queue.
@agrare agrare force-pushed the fix_miq_request_task_resource_action_workflow branch 3 times, most recently from dc524e8 to 15a393e Compare August 26, 2024 16:46
@agrare agrare changed the title [WIP] Fix MiqRequestTask ResourceAction with a Workflow Fix MiqRequestTask ResourceAction with a Workflow Aug 26, 2024
@miq-bot miq-bot removed the wip label Aug 26, 2024
@agrare agrare force-pushed the fix_miq_request_task_resource_action_workflow branch from 15a393e to 9b75538 Compare August 26, 2024 17:43
@agrare agrare force-pushed the fix_miq_request_task_resource_action_workflow branch from 9b75538 to c69f814 Compare August 26, 2024 17:44
@agrare agrare force-pushed the fix_miq_request_task_resource_action_workflow branch from 3ca43d5 to aa5b237 Compare August 26, 2024 19:42
@miq-bot
Copy link
Member

miq-bot commented Aug 26, 2024

Checked commits agrare/manageiq@b489fe3~...aa5b237 with ruby 3.1.5, rubocop 1.56.3, haml-lint 0.51.0, and yamllint
4 files checked, 0 offenses detected
Everything looks fine. 🏆

@kbrock kbrock merged commit 4b49d30 into ManageIQ:master Aug 27, 2024
8 checks passed
@agrare agrare deleted the fix_miq_request_task_resource_action_workflow branch August 27, 2024 20:10
@Fryguy
Copy link
Member

Fryguy commented Aug 28, 2024

Backported to radjabov in commit 1286aa3.

commit 1286aa34992340cad7a0778df01a845f1772e561
Author: Keenan Brock <[email protected]>
Date:   Tue Aug 27 14:37:45 2024 -0400

    Merge pull request #23123 from agrare/fix_miq_request_task_resource_action_workflow
    
    Fix MiqRequestTask ResourceAction with a Workflow
    
    (cherry picked from commit 4b49d3055eb87173cc8c7b983b8ebf7500737989)

Fryguy pushed a commit that referenced this pull request Aug 28, 2024
…ction_workflow

Fix MiqRequestTask ResourceAction with a Workflow

(cherry picked from commit 4b49d30)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants