Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rubygems for profiling and performance analyzing #9953

Open
wants to merge 1 commit into
base: rpm/develop
Choose a base branch
from
Open

Add rubygems for profiling and performance analyzing #9953

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 13 additions & 7 deletions comps/comps-foreman-el8.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@
<packagereq type="default">rubygem-mail</packagereq>
<packagereq type="default">rubygem-marcel</packagereq>
<packagereq type="default">rubygem-memoist</packagereq>
<packagereq type="default">rubygem-memory_profiler</packagereq>
<packagereq type="default">rubygem-method_source</packagereq>
<packagereq type="default">rubygem-mime-types</packagereq>
<packagereq type="default">rubygem-mime-types-data</packagereq>
Expand Down Expand Up @@ -248,6 +249,7 @@
<packagereq type="default">rubygem-rack</packagereq>
<packagereq type="default">rubygem-rack-cors</packagereq>
<packagereq type="default">rubygem-rack-jsonp</packagereq>
<packagereq type="default">rubygem-rack-mini-profiler</packagereq>
<packagereq type="default">rubygem-rack-protection</packagereq>
<packagereq type="default">rubygem-rack-test</packagereq>
<packagereq type="default">rubygem-rails</packagereq>
Expand Down Expand Up @@ -292,6 +294,7 @@
<packagereq type="default">rubygem-sprockets-rails</packagereq>
<packagereq type="default">rubygem-sqlite3</packagereq>
<packagereq type="default">rubygem-sshkey</packagereq>
<packagereq type="default">rubygem-stackprof</packagereq>
<packagereq type="default">rubygem-statsd-instrument</packagereq>
<packagereq type="default">rubygem-terser</packagereq>
<packagereq type="default">rubygem-text</packagereq>
Expand Down Expand Up @@ -404,8 +407,8 @@
<packagereq type="default">rubygem-google-apis-pubsub_v1-doc</packagereq>
<packagereq type="default">rubygem-google-apis-sqladmin_v1beta4-doc</packagereq>
<packagereq type="default">rubygem-google-apis-storage_v1-doc</packagereq>
<packagereq type="default">rubygem-googleauth-doc</packagereq>
<packagereq type="default">rubygem-google-cloud-env-doc</packagereq>
<packagereq type="default">rubygem-googleauth-doc</packagereq>
<packagereq type="default">rubygem-graphql-batch-doc</packagereq>
<packagereq type="default">rubygem-graphql-doc</packagereq>
<packagereq type="default">rubygem-gssapi-doc</packagereq>
Expand All @@ -415,8 +418,8 @@
<packagereq type="default">rubygem-highline-doc</packagereq>
<packagereq type="default">rubygem-hocon-doc</packagereq>
<packagereq type="default">rubygem-http-accept-doc</packagereq>
<packagereq type="default">rubygem-httpclient-doc</packagereq>
<packagereq type="default">rubygem-http-cookie-doc</packagereq>
<packagereq type="default">rubygem-httpclient-doc</packagereq>
<packagereq type="default">rubygem-i18n-doc</packagereq>
<packagereq type="default">rubygem-ipaddress-doc</packagereq>
<packagereq type="default">rubygem-journald-logger-doc</packagereq>
Expand All @@ -435,6 +438,7 @@
<packagereq type="default">rubygem-mail-doc</packagereq>
<packagereq type="default">rubygem-marcel-doc</packagereq>
<packagereq type="default">rubygem-memoist-doc</packagereq>
<packagereq type="default">rubygem-memory_profiler-doc</packagereq>
<packagereq type="default">rubygem-method_source-doc</packagereq>
<packagereq type="default">rubygem-mime-types-data-doc</packagereq>
<packagereq type="default">rubygem-mime-types-doc</packagereq>
Expand All @@ -443,12 +447,12 @@
<packagereq type="default">rubygem-multi_json-doc</packagereq>
<packagereq type="default">rubygem-multipart-post-doc</packagereq>
<packagereq type="default">rubygem-mustermann-doc</packagereq>
<packagereq type="default">rubygem-net_http_unix-doc</packagereq>
<packagereq type="default">rubygem-net-ldap-doc</packagereq>
<packagereq type="default">rubygem-net-ping-doc</packagereq>
<packagereq type="default">rubygem-netrc-doc</packagereq>
<packagereq type="default">rubygem-net-scp-doc</packagereq>
<packagereq type="default">rubygem-net-ssh-doc</packagereq>
<packagereq type="default">rubygem-net_http_unix-doc</packagereq>
<packagereq type="default">rubygem-netrc-doc</packagereq>
<packagereq type="default">rubygem-nio4r-doc</packagereq>
<packagereq type="default">rubygem-nokogiri-doc</packagereq>
<packagereq type="default">rubygem-oauth-doc</packagereq>
Expand All @@ -471,6 +475,7 @@
<packagereq type="default">rubygem-rack-cors-doc</packagereq>
<packagereq type="default">rubygem-rack-doc</packagereq>
<packagereq type="default">rubygem-rack-jsonp-doc</packagereq>
<packagereq type="default">rubygem-rack-mini-profiler-doc</packagereq>
<packagereq type="default">rubygem-rack-protection-doc</packagereq>
<packagereq type="default">rubygem-rack-test-doc</packagereq>
<packagereq type="default">rubygem-rails-doc</packagereq>
Expand All @@ -492,15 +497,15 @@
<packagereq type="default">rubygem-roadie-doc</packagereq>
<packagereq type="default">rubygem-roadie-rails-doc</packagereq>
<packagereq type="default">rubygem-rsec-doc</packagereq>
<packagereq type="default">rubygem-ruby-libvirt-doc</packagereq>
<packagereq type="default">rubygem-ruby2_keywords-doc</packagereq>
<packagereq type="default">rubygem-ruby2ruby-doc</packagereq>
<packagereq type="default">rubygem-rubyipmi-doc</packagereq>
<packagereq type="default">rubygem-ruby-libvirt-doc</packagereq>
<packagereq type="default">rubygem-ruby_parser-doc</packagereq>
<packagereq type="default">rubygem-rubyipmi-doc</packagereq>
<packagereq type="default">rubygem-safemode-doc</packagereq>
<packagereq type="default">rubygem-sass-rails-doc</packagereq>
<packagereq type="default">rubygem-sassc-doc</packagereq>
<packagereq type="default">rubygem-sassc-rails-doc</packagereq>
<packagereq type="default">rubygem-sass-rails-doc</packagereq>
<packagereq type="default">rubygem-scoped_search-doc</packagereq>
<packagereq type="default">rubygem-sd_notify-doc</packagereq>
<packagereq type="default">rubygem-secure_headers-doc</packagereq>
Expand All @@ -515,6 +520,7 @@
<packagereq type="default">rubygem-sprockets-rails-doc</packagereq>
<packagereq type="default">rubygem-sqlite3-doc</packagereq>
<packagereq type="default">rubygem-sshkey-doc</packagereq>
<packagereq type="default">rubygem-stackprof-doc</packagereq>
<packagereq type="default">rubygem-statsd-instrument-doc</packagereq>
<packagereq type="default">rubygem-terser-doc</packagereq>
<packagereq type="default">rubygem-text-doc</packagereq>
Expand Down
14 changes: 7 additions & 7 deletions comps/comps-foreman-plugins-el8.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,7 @@
<packagereq type="default">rubygem-ffi-compiler-doc</packagereq>
<packagereq type="default">rubygem-fog-kubevirt-doc</packagereq>
<packagereq type="default">rubygem-fog-proxmox-doc</packagereq>
<packagereq type="default">rubygem-foreman-tasks-doc</packagereq>
<packagereq type="default">rubygem-foreman_acd-doc</packagereq>
<packagereq type="default">rubygem-foreman_ansible-doc</packagereq>
<packagereq type="default">rubygem-foreman_azure_rm-doc</packagereq>
Expand Down Expand Up @@ -270,7 +271,6 @@
<packagereq type="default">rubygem-foreman_snapshot_management-doc</packagereq>
<packagereq type="default">rubygem-foreman_statistics-doc</packagereq>
<packagereq type="default">rubygem-foreman_supervisory_authority-doc</packagereq>
<packagereq type="default">rubygem-foreman-tasks-doc</packagereq>
<packagereq type="default">rubygem-foreman_templates-doc</packagereq>
<packagereq type="default">rubygem-foreman_vault-doc</packagereq>
<packagereq type="default">rubygem-foreman_vmwareannotations-doc</packagereq>
Expand All @@ -279,14 +279,14 @@
<packagereq type="default">rubygem-fugit-doc</packagereq>
<packagereq type="default">rubygem-gapic-common-doc</packagereq>
<packagereq type="default">rubygem-git-doc</packagereq>
<packagereq type="default">rubygem-googleapis-common-protos-doc</packagereq>
<packagereq type="default">rubygem-googleapis-common-protos-types-doc</packagereq>
<packagereq type="default">rubygem-google-cloud-common-doc</packagereq>
<packagereq type="default">rubygem-google-cloud-compute-doc</packagereq>
<packagereq type="default">rubygem-google-cloud-compute-v1-doc</packagereq>
<packagereq type="default">rubygem-google-cloud-core-doc</packagereq>
<packagereq type="default">rubygem-google-cloud-errors-doc</packagereq>
<packagereq type="default">rubygem-google-protobuf-doc</packagereq>
<packagereq type="default">rubygem-googleapis-common-protos-doc</packagereq>
<packagereq type="default">rubygem-googleapis-common-protos-types-doc</packagereq>
<packagereq type="default">rubygem-graphite-api-doc</packagereq>
<packagereq type="default">rubygem-grpc-doc</packagereq>
<packagereq type="default">rubygem-hammer_cli_foreman_admin-doc</packagereq>
Expand All @@ -305,10 +305,10 @@
<packagereq type="default">rubygem-hammer_cli_foreman_tasks-doc</packagereq>
<packagereq type="default">rubygem-hammer_cli_foreman_templates-doc</packagereq>
<packagereq type="default">rubygem-hammer_cli_foreman_webhooks-doc</packagereq>
<packagereq type="default">rubygem-httparty-doc</packagereq>
<packagereq type="default">rubygem-http-doc</packagereq>
<packagereq type="default">rubygem-http-form_data-doc</packagereq>
<packagereq type="default">rubygem-http_parser.rb-doc</packagereq>
<packagereq type="default">rubygem-httparty-doc</packagereq>
<packagereq type="default">rubygem-infoblox-doc</packagereq>
<packagereq type="default">rubygem-interactor-doc</packagereq>
<packagereq type="default">rubygem-jgrep-doc</packagereq>
Expand All @@ -317,12 +317,12 @@
<packagereq type="default">rubygem-kubeclient-doc</packagereq>
<packagereq type="default">rubygem-logify-doc</packagereq>
<packagereq type="default">rubygem-mqtt-doc</packagereq>
<packagereq type="default">rubygem-ms_rest_azure-doc</packagereq>
<packagereq type="default">rubygem-ms_rest-doc</packagereq>
<packagereq type="default">rubygem-ms_rest_azure-doc</packagereq>
<packagereq type="default">rubygem-multi_xml-doc</packagereq>
<packagereq type="default">rubygem-netbox-client-ruby-doc</packagereq>
<packagereq type="default">rubygem-net-ssh-gateway-doc</packagereq>
<packagereq type="default">rubygem-net-ssh-multi-doc</packagereq>
<packagereq type="default">rubygem-netbox-client-ruby-doc</packagereq>
<packagereq type="default">rubygem-newt-doc</packagereq>
<packagereq type="default">rubygem-openscap-doc</packagereq>
<packagereq type="default">rubygem-openscap_parser-doc</packagereq>
Expand All @@ -343,6 +343,7 @@
<packagereq type="default">rubygem-ruby-hmac-doc</packagereq>
<packagereq type="default">rubygem-ruby-nmap-doc</packagereq>
<packagereq type="default">rubygem-sentry-raven-doc</packagereq>
<packagereq type="default">rubygem-smart-proxy-probing-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_acd-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_ansible-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_container_gateway-doc</packagereq>
Expand All @@ -362,7 +363,6 @@
<packagereq type="default">rubygem-smart_proxy_monitoring-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_omaha-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_openscap-doc</packagereq>
<packagereq type="default">rubygem-smart-proxy-probing-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_pulp-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_realm_ad_plugin-doc</packagereq>
<packagereq type="default">rubygem-smart_proxy_remote_execution_ssh-doc</packagereq>
Expand Down
3 changes: 3 additions & 0 deletions package_manifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,7 @@ ruby_core_packages_tier1:
rubygem-mail: {}
rubygem-marcel: {}
rubygem-memoist: {}
rubygem-memory_profiler: {}
rubygem-method_source: {}
rubygem-mime-types: {}
rubygem-mime-types-data: {}
Expand Down Expand Up @@ -380,6 +381,7 @@ ruby_core_packages_tier1:
rubygem-rack: {}
rubygem-rack-cors: {}
rubygem-rack-jsonp: {}
rubygem-rack-mini-profiler: {}
rubygem-rack-protection: {}
rubygem-rack-test: {}
rubygem-rails-dom-testing: {}
Expand Down Expand Up @@ -414,6 +416,7 @@ ruby_core_packages_tier1:
rubygem-sprockets: {}
rubygem-sprockets-rails: {}
rubygem-sshkey: {}
rubygem-stackprof: {}
rubygem-statsd-instrument: {}
rubygem-sqlite3: {}
rubygem-terser: {}
Expand Down
24 changes: 23 additions & 1 deletion packages/foreman/foreman/foreman.spec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
%global dynflow_sidekiq_service_name dynflow-sidekiq@
%global rake /usr/bin/rake

%global release 2
%global release 3
%global prereleasesource develop
%global prerelease %{?prereleasesource}

Expand Down Expand Up @@ -527,6 +527,22 @@ Configuration files for the Performance Co-Pilot integration
%{_sysconfdir}/pcp/proc/%{name}-hotproc.conf
%{_sharedstatedir}/pcp/config/pmlogconf/%{name}-hotproc

%package profiling
Summary: Adds rails-profiling extensions to foreman
# start specfile profiling Requires
Requires: rubygem(rack-mini-profiler)
Requires: rubygem(memory_profiler)
Requires: rubygem(stackprof)
# end specfile profiling Requires

%description profiling
This automatically installs and enables profiling extensions to foreman.
It is probably not what you want on a productive system.

%files profiling
%{_datadir}/%{name}/bundler.d/profiling.rb
%{_datadir}/%{name}/config/initializers/rack_mini_profiler.rb
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think by default this means the file is present in both foreman and foreman-profiling. Be sure to check that.

It may also be fine to include it in core if it gracefully falls back if the dependencies aren't present. Though there may be a security concern so I'm not strict on that. Just want to make sure you're doing what you intend to do.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what I can say from trying it out is that in 'productive-mode, which means Gemfile.In` is present, all available groups are loaded, regardless of it being optional or not.

When we use bundler, groups matter, but even though I do bundle --with profiling, the group would have to be required somewhere in code like here: https://github.com/theforeman/foreman/blob/fa79806973c6de08b6d265ed1ae99c0ff4f06bd6/config/application.rb#L59

Assuming we want to have it in development-environment and in production, if it is installed, then I would keep it in bundler.d/profiling.rb but add it to development-group.

I think by default this means the file is present in both foreman and foreman-profiling. Be sure to check that.

AFAIK having the same filepath in multiple packages is a big NoNo in most packaging-frameworks and my first test suggests it is only in the foreman-profiling package after build.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what I can say from trying it out is that in productive-mode, which means Gemfile.In is present, all available groups are loaded, regardless of it being optional or not.

That's due to bundler_ext, which we use on Red Hat. We don't on Debian. Perhaps this is a good excuse to drop bundler_ext usage. The original use case was that you could ignore gem dependencies in RPMs, for security patching. These days we just make sure the gem dependencies are correct.

AFAIK having the same filepath in multiple packages is a big NoNo in most packaging-frameworks and my first test suggests it is only in the foreman-profiling package after build.

I know I've seen weirdness with this in the past, so just wanted to highlight that it should be checked.


%description
Foreman is aimed to be a Single Address For All Machines Life Cycle Management.
Foreman is based on Ruby on Rails, and this package bundles Rails and all
Expand Down Expand Up @@ -559,6 +575,9 @@ export NODE_ENV=production
%{rake} assets:precompile RAILS_ENV=production DATABASE_URL=nulldb://nohost --trace
rm db/schema.rb

# enable profiling if the bundler.d file is present
sed -i 's/,\s*optional:\s*true//' bundler.d/profiling.rb

%install
rm -rf %{buildroot}

Expand Down Expand Up @@ -861,6 +880,9 @@ exit 0
%systemd_postun %{name}.socket

%changelog
* Mon Apr 08 2024 Markus Bucher <[email protected]> - 3.11.0-0.3.develop
- Add profiling subpackage

* Mon Mar 04 2024 Evgeni Golov - 3.11.0-0.2.develop
- Update GEM Requiremens

Expand Down
1 change: 1 addition & 0 deletions packages/foreman/rubygem-memory_profiler/memory_profiler-1.0.1.gem
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../../../.git/annex/objects/qx/Z5/SHA256E-s16384--38cdb42f22d9100df2eba0365c199724b58b05c38e765cd764a07392916901b1.1.gem/SHA256E-s16384--38cdb42f22d9100df2eba0365c199724b58b05c38e765cd764a07392916901b1.1.gem
70 changes: 70 additions & 0 deletions packages/foreman/rubygem-memory_profiler/rubygem-memory_profiler.spec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# template: default
%global gem_name memory_profiler

Name: rubygem-%{gem_name}
Version: 1.0.1
Release: 1%{?dist}
Summary: Memory profiling routines for Ruby 2.5+
License: MIT
URL: https://github.com/SamSaffron/memory_profiler
Source0: https://rubygems.org/gems/%{gem_name}-%{version}.gem

# start specfile generated dependencies
Requires: ruby >= 2.5.0
BuildRequires: ruby >= 2.5.0
BuildRequires: rubygems-devel
BuildArch: noarch
# end specfile generated dependencies

%description
Memory profiling routines for Ruby 2.5+.


%package doc
Summary: Documentation for %{name}
Requires: %{name} = %{version}-%{release}
BuildArch: noarch

%description doc
Documentation for %{name}.

%prep
%setup -q -n %{gem_name}-%{version}

%build
# Create the gem as gem install only works on a gem file
gem build ../%{gem_name}-%{version}.gemspec

# %%gem_install compiles any C extensions and installs the gem into ./%%gem_dir
# by default, so that we can move it into the buildroot in %%install
%gem_install

%install
mkdir -p %{buildroot}%{gem_dir}
cp -a .%{gem_dir}/* \
%{buildroot}%{gem_dir}/

mkdir -p %{buildroot}%{_bindir}
cp -a .%{_bindir}/* \
%{buildroot}%{_bindir}/

find %{buildroot}%{gem_instdir}/bin -type f | xargs chmod a+x

%files
%dir %{gem_instdir}
%{_bindir}/ruby-memory-profiler
%license %{gem_instdir}/LICENSE.txt
%{gem_instdir}/bin
%{gem_libdir}
%exclude %{gem_cache}
%{gem_spec}

%files doc
%doc %{gem_docdir}
%doc %{gem_instdir}/CHANGELOG.md
%doc %{gem_instdir}/README.md

%changelog
* Mon Nov 06 2023 Markus Bucher <[email protected]> 1.0.1-1
- Add rubygem-memory_profiler generated by gem2rpm using the default template

1 change: 1 addition & 0 deletions packages/foreman/rubygem-rack-mini-profiler/rack-mini-profiler-3.1.1.gem
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../../../.git/annex/objects/1W/4M/SHA256E-s412672--3bbcb84abdfe51e05341a58d6714e027499f974245c4738c74ee38400e3e72a3.1.gem/SHA256E-s412672--3bbcb84abdfe51e05341a58d6714e027499f974245c4738c74ee38400e3e72a3.1.gem
63 changes: 63 additions & 0 deletions packages/foreman/rubygem-rack-mini-profiler/rubygem-rack-mini-profiler.spec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# template: default
%global gem_name rack-mini-profiler

Name: rubygem-%{gem_name}
Version: 3.1.1
Release: 1%{?dist}
Summary: Profiles loading speed for rack applications
License: MIT
URL: https://miniprofiler.com
Source0: https://rubygems.org/gems/%{gem_name}-%{version}.gem

# start specfile generated dependencies
Requires: ruby >= 2.6.0
BuildRequires: ruby >= 2.6.0
BuildRequires: rubygems-devel
BuildArch: noarch
# end specfile generated dependencies

%description
Profiling toolkit for Rack applications with Rails integration. Client Side
profiling, DB profiling and Server profiling.


%package doc
Summary: Documentation for %{name}
Requires: %{name} = %{version}-%{release}
BuildArch: noarch

%description doc
Documentation for %{name}.

%prep
%setup -q -n %{gem_name}-%{version}

%build
# Create the gem as gem install only works on a gem file
gem build ../%{gem_name}-%{version}.gemspec

# %%gem_install compiles any C extensions and installs the gem into ./%%gem_dir
# by default, so that we can move it into the buildroot in %%install
%gem_install

%install
mkdir -p %{buildroot}%{gem_dir}
cp -a .%{gem_dir}/* \
%{buildroot}%{gem_dir}/

%files
%dir %{gem_instdir}
%{gem_libdir}
%exclude %{gem_cache}
%{gem_spec}

%files doc
%doc %{gem_docdir}
%doc %{gem_instdir}/CHANGELOG.md
%doc %{gem_instdir}/README.md
%exclude %{gem_instdir}/rack-mini-profiler.gemspec

%changelog
* Mon Nov 06 2023 Markus Bucher <[email protected]> 3.1.1-1
- Add rubygem-rack-mini-profiler generated by gem2rpm using the default template

Loading