Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[dev.icinga.com #13409] Crash in CreateObjectHandler (regression from #11684 #4828

Closed
icinga-migration opened this issue Dec 5, 2016 · 6 comments
Closed

[dev.icinga.com #13409] Crash in CreateObjectHandler (regression from #11684 #4828

icinga-migration opened this issue Dec 5, 2016 · 6 comments
Labels
area/api REST API bug Something isn't working
Milestone
2.6.0

Comments

@icinga-migration
Copy link

This issue has been migrated from Redmine: https://dev.icinga.com/issues/13409

Created by magnus on 2016-12-05 15:21:13 00:00

Assignee: mfriedrich
Status: Resolved (closed on 2016-12-05 15:38:29 00:00)
Target Version: 2.6.0
Last Update: 2016-12-07 08:24:58 00:00 (in Redmine)

Icinga Version: 2.5.4 snapshot2016.12.04
Backport?: Not yet backported
Include in Changelog: 0

Hello,

I did an upgrade to the latest snapshot since I am affected by the api related crash which I filed a bug for earlier.

This snapshot crashes even quicker.

I attach some information below, let me know if you need anything else:

  1. dpkg -l |grep icinga
    ii icinga2 2.5.4 snapshot2016.12.04 1xenial amd64 host and network monitoring system
    ii icinga2-bin 2.5.4 snapshot2016.12.04 1    xenial amd64 host and network monitoring system - daemon
    ii icinga2-common 2.5.4 snapshot2016.12.04 1xenial all host and network monitoring system - common files
    ii icinga2-doc 2.5.4 snapshot2016.12.04 1    xenial all host and network monitoring system - documentation
    ii libicinga2 2.5.4 snapshot2016.12.04 1xenial amd64 host and network monitoring system - internal libraries
    ii vim-icinga2 2.5.4 snapshot2016.12.04 1    xenial all syntax highlighting for Icinga 2 config files in VIM
  1. cat /etc/os-release
    NAME="Ubuntu"
    VERSION="16.04.1 LTS (Xenial Xerus)"
    ID=ubuntu
    ID_LIKE=debian
    PRETTY_NAME="Ubuntu 16.04.1 LTS"
    VERSION_ID="16.04"
    HOME_URL="http://www.ubuntu.com/"
    SUPPORT_URL="http://help.ubuntu.com/"
    BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
    VERSION_CODENAME=xenial
    UBUNTU_CODENAME=xenial
  1. cat report.1480931959.719613
    Application information:
    Application version: v2.5.4-213-gd076617
    Installation root: /usr
    Sysconf directory: /etc
    Run directory: /run
    Local state directory: /var
    Package data directory: /usr/share/icinga2
    State path: /var/lib/icinga2/icinga2.state
    Modified attributes path: /var/lib/icinga2/modified-attributes.conf
    Objects path: /var/cache/icinga2/icinga2.debug
    Vars path: /var/cache/icinga2/icinga2.vars
    PID path: /run/icinga2/icinga2.pid

System information:
Platform: Ubuntu
Platform version: 16.04.1 LTS (Xenial Xerus)
Kernel: Linux
Kernel version: 4.4.0-21-generic
Architecture: x86_64

Build information:
Compiler: GNU 5.3.1
Build host: 65a501d4290c
Stacktrace:

(0) libpthread.so.0: ( 0x113e0) [0x7f863aee33e0]
(1) libc.so.6: gsignal ( 0x38) [0x7f8639c96428]
(2) libc.so.6: abort ( 0x16a) [0x7f8639c9802a]
(3) libc.so.6: ( 0x2dbd7) [0x7f8639c8ebd7]
(4) libc.so.6: ( 0x2dc82) [0x7f8639c8ec82]
(5) libremote.so.2.5.4: icinga::CreateObjectHandler::HandleRequest(boost::intrusive_ptricinga::ApiUser const&, icinga::HttpRequest&, icinga::HttpResponse&, boost::intrusive_ptricinga::Dictionary const&) ( 0x10b7) [0x7f8639475de7]
(6) libremote.so.2.5.4: icinga::HttpHandler::ProcessRequest(boost::intrusive_ptricinga::ApiUser const&, icinga::HttpRequest&, icinga::HttpResponse&) ( 0x68c) [0x7f8639464e0c]
(7) libremote.so.2.5.4: icinga::HttpServerConnection::ProcessMessageAsync(icinga::HttpRequest&) ( 0x54f) [0x7f8639465a0f]
(8) libbase.so.2.5.4: icinga::WorkQueue::WorkerThreadProc() ( 0x529) [0x7f863a631939]
(9) libboost_thread.so.1.58.0: ( 0x115d5) [0x7f863b5825d5]
(10) libpthread.so.0: ( 0x770a) [0x7f863aed970a]
(11) libc.so.6: clone ( 0x6d) [0x7f8639d6782d]

  • This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
  • please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
  • information that might be useful in order to reproduce this problem.
    *****
  1. cat report.1480940907.376057
    Application information:
    Application version: v2.5.4-213-gd076617
    Installation root: /usr
    Sysconf directory: /etc
    Run directory: /run
    Local state directory: /var
    Package data directory: /usr/share/icinga2
    State path: /var/lib/icinga2/icinga2.state
    Modified attributes path: /var/lib/icinga2/modified-attributes.conf
    Objects path: /var/cache/icinga2/icinga2.debug
    Vars path: /var/cache/icinga2/icinga2.vars
    PID path: /run/icinga2/icinga2.pid

System information:
Platform: Ubuntu
Platform version: 16.04.1 LTS (Xenial Xerus)
Kernel: Linux
Kernel version: 4.4.0-21-generic
Architecture: x86_64

Build information:
Compiler: GNU 5.3.1
Build host: 65a501d4290c
Stacktrace:

(0) libpthread.so.0: ( 0x113e0) [0x7f5b1f3413e0]
(1) libc.so.6: gsignal ( 0x38) [0x7f5b1e0f4428]
(2) libc.so.6: abort ( 0x16a) [0x7f5b1e0f602a]
(3) libc.so.6: ( 0x2dbd7) [0x7f5b1e0ecbd7]
(4) libc.so.6: ( 0x2dc82) [0x7f5b1e0ecc82]
(5) libremote.so.2.5.4: icinga::CreateObjectHandler::HandleRequest(boost::intrusive_ptricinga::ApiUser const&, icinga::HttpRequest&, icinga::HttpResponse&, boost::intrusive_ptricinga::Dictionary const&) ( 0x10b7) [0x7f5b1d8d3de7]
(6) libremote.so.2.5.4: icinga::HttpHandler::ProcessRequest(boost::intrusive_ptricinga::ApiUser const&, icinga::HttpRequest&, icinga::HttpResponse&) ( 0x68c) [0x7f5b1d8c2e0c]
(7) libremote.so.2.5.4: icinga::HttpServerConnection::ProcessMessageAsync(icinga::HttpRequest&) ( 0x54f) [0x7f5b1d8c3a0f]
(8) libbase.so.2.5.4: icinga::WorkQueue::WorkerThreadProc() ( 0x529) [0x7f5b1ea8f939]
(9) libboost_thread.so.1.58.0: ( 0x115d5) [0x7f5b1f9e05d5]
(10) libpthread.so.0: ( 0x770a) [0x7f5b1f33770a]
(11) libc.so.6: clone ( 0x6d) [0x7f5b1e1c582d]

  • This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
  • please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
  • information that might be useful in order to reproduce this problem.
    *****
  1. cat report.1480943895.836775
    Application information:
    Application version: v2.5.4-214-ge954a0e
    Installation root: /usr
    Sysconf directory: /etc
    Run directory: /run
    Local state directory: /var
    Package data directory: /usr/share/icinga2
    State path: /var/lib/icinga2/icinga2.state
    Modified attributes path: /var/lib/icinga2/modified-attributes.conf
    Objects path: /var/cache/icinga2/icinga2.debug
    Vars path: /var/cache/icinga2/icinga2.vars
    PID path: /run/icinga2/icinga2.pid

System information:
Platform: Ubuntu
Platform version: 16.04.1 LTS (Xenial Xerus)
Kernel: Linux
Kernel version: 4.4.0-21-generic
Architecture: x86_64

Build information:
Compiler: GNU 5.3.1
Build host: a987a60f2c36
Stacktrace:

(0) libpthread.so.0: ( 0x113e0) [0x7f9b0ce873e0]
(1) libc.so.6: gsignal ( 0x38) [0x7f9b0bc3a428]
(2) libc.so.6: abort ( 0x16a) [0x7f9b0bc3c02a]
(3) libc.so.6: ( 0x2dbd7) [0x7f9b0bc32bd7]
(4) libc.so.6: ( 0x2dc82) [0x7f9b0bc32c82]
(5) libremote.so.2.5.4: icinga::CreateObjectHandler::HandleRequest(boost::intrusive_ptricinga::ApiUser const&, icinga::HttpRequest&, icinga::HttpResponse&, boost::intrusive_ptricinga::Dictionary const&) ( 0x10b7) [0x7f9b0b419de7]
(6) libremote.so.2.5.4: icinga::HttpHandler::ProcessRequest(boost::intrusive_ptricinga::ApiUser const&, icinga::HttpRequest&, icinga::HttpResponse&) ( 0x68c) [0x7f9b0b408e0c]
(7) libremote.so.2.5.4: icinga::HttpServerConnection::ProcessMessageAsync(icinga::HttpRequest&) ( 0x54f) [0x7f9b0b409a0f]
(8) libbase.so.2.5.4: icinga::WorkQueue::WorkerThreadProc() ( 0x529) [0x7f9b0c5d5939]
(9) libboost_thread.so.1.58.0: ( 0x115d5) [0x7f9b0d5265d5]
(10) libpthread.so.0: ( 0x770a) [0x7f9b0ce7d70a]
(11) libc.so.6: clone ( 0x6d) [0x7f9b0bd0b82d]

  • This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
  • please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
  • information that might be useful in order to reproduce this problem.
    *****

Changesets

2016-12-05 15:37:31 00:00 by mfriedrich 338f5c0

Fix crash in CreateObjectHandler::HandleRequest()

fixes #13409
refs #11684

2016-12-06 07:57:27 00:00 by gbeutner fe12a10

Remove redundant check

refs #13409
@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-12-05 15:26:25 00:00

  • Category set to API
  • Status changed from New to Feedback
  • Assigned to set to magnus

Which PUT request are you firing against the Icinga 2 API? By chance, an empty "attrs" field in the body?

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-12-05 15:38:03 00:00

  • Subject changed from Icinga crashes to Crash in CreateObjectHandler (regression from #11684
  • Status changed from Feedback to Assigned
  • Target Version set to 2.6.0
  • Include in Changelog changed from 1 to 0

Ok, that's coming from a patch in 099fc76

Thanks for testing!

michi@mbmif ~ $ curl -k -s -u root:icinga -H 'Accept: application/json' -X PUT 'https://localhost:5665/v1/objects/hosts/db-server-01' -d '{ "templates": [ "generic-host" ] }'

mbmif /usr/local/icinga2 (master) # lldb -- /usr/local/icinga2/lib/icinga2/sbin/icinga2 daemon
(lldb) target create "/usr/local/icinga2/lib/icinga2/sbin/icinga2"
Current executable set to '/usr/local/icinga2/lib/icinga2/sbin/icinga2' (x86_64).
(lldb) settings set -- target.run-args  "daemon"
(lldb) r
Process 17998 launched: '/usr/local/icinga2/lib/icinga2/sbin/icinga2' (x86_64)
[2016-12-05 16:26:59  0100] information/cli: Icinga application loader (version: v2.5.4-213-gd076617; debug)
[2016-12-05 16:26:59  0100] information/cli: Loading configuration file(s).
[2016-12-05 16:27:00  0100] information/ConfigItem: Committing config item(s).
[2016-12-05 16:27:00  0100] information/ApiListener: My API identity: mbmif.int.netways.de
[2016-12-05 16:27:00  0100] warning/ApplyRule: Apply rule 'satellite-host' (in /usr/local/icinga2/etc/icinga2/conf.d/satellite.conf: 29:1-29:41) for type 'Dependency' does not match anywhere!
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 2 FileLoggers.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 2 Endpoints.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 3 Zones.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 ApiListener.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 3 ApiUsers.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 203 CheckCommands.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 145 Comments.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 13 Services.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 3 ServiceGroups.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 3 TimePeriods.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 User.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 UserGroup.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 5 Hosts.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 2 HostGroups.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 IcingaApplication.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 2 NotificationCommands.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 12 Notifications.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 CheckerComponent.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 NotificationComponent.
[2016-12-05 16:27:00  0100] information/ScriptGlobal: Dumping variables to file '/usr/local/icinga2/var/cache/icinga2/icinga2.vars'
[2016-12-05 16:27:00  0100] information/ConfigObject: Restoring program state from file '/usr/local/icinga2/var/lib/icinga2/icinga2.state'
[2016-12-05 16:27:00  0100] information/ConfigObject: Restored 408 objects. Loaded 0 new objects without state.
[2016-12-05 16:27:00  0100] information/ConfigItem: Triggering Start signal for config items
[2016-12-05 16:27:00  0100] information/ApiListener: Copying 3 zone configuration files for zone 'global-templates' to '/usr/local/icinga2/var/lib/icinga2/api/zones/global-templates'.
[2016-12-05 16:27:00  0100] information/ApiListener: Adding new listener on port '5665'
[2016-12-05 16:27:00  0100] information/ConfigItem: Activated all objects.
[2016-12-05 16:27:00  0100] information/ConfigItem: Committing config item(s).
[2016-12-05 16:27:00  0100] information/DbConnection: Resuming IDO connection: ido-mysql
[2016-12-05 16:27:00  0100] warning/ApplyRule: Apply rule 'satellite-host' (in /usr/local/icinga2/etc/icinga2/conf.d/satellite.conf: 29:1-29:41) for type 'Dependency' does not match anywhere!
[2016-12-05 16:27:00  0100] information/ConfigItem: Instantiated 1 Downtime.
[2016-12-05 16:27:00  0100] information/ConfigItem: Triggering Start signal for config items
[2016-12-05 16:27:00  0100] information/ConfigItem: Activated all objects.
[2016-12-05 16:27:00  0100] information/IdoMysqlConnection: MySQL IDO instance id: 1 (schema version: '1.14.2')
[2016-12-05 16:27:00  0100] information/IdoMysqlConnection: Finished reconnecting to MySQL IDO database in 0.743363 second(s).
[2016-12-05 16:27:15  0100] information/IdoMysqlConnection: Query queue items: 0, query rate: 8.56667/s (514/min 514/5min 514/15min);
[2016-12-05 16:27:26  0100] information/ApiListener: New client connection from [127.0.0.1]:61708 (no client certificate)
[2016-12-05 16:27:26  0100] information/HttpServerConnection: Request: PUT /v1/objects/hosts/db-server-01 (from [127.0.0.1]:61708, user: root)
Assertion failed: (px != 0), function operator->, file /usr/local/include/boost/smart_ptr/intrusive_ptr.hpp, line 198.
Process 17998 stopped
* thread #8: tid = 0x282346, 0x00007fff9a835dda libsystem_kernel.dylib`__pthread_kill   10, stop reason = signal SIGABRT
    frame #0: 0x00007fff9a835dda libsystem_kernel.dylib`__pthread_kill   10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff9a835dda < 10>: jae    0x7fff9a835de4            ; < 20>
    0x7fff9a835ddc < 12>: movq   %rax, %rdi
    0x7fff9a835ddf < 15>: jmp    0x7fff9a82ed6f            ; cerror_nocancel
    0x7fff9a835de4 < 20>: retq
(lldb) up
frame #1: 0x00007fff9a921787 libsystem_pthread.dylib`pthread_kill   90
libsystem_pthread.dylib`pthread_kill:
    0x7fff9a921787 < 90>:  cmpl   $-0x1, �x
    0x7fff9a92178a < 93>:  jne    0x7fff9a921793            ; < 102>
    0x7fff9a92178c < 95>:  callq  0x7fff9a9231fa            ; symbol stub for: __error
    0x7fff9a921791 < 100>: movl   (%rax), �x
(lldb)
frame #2: 0x00007fff9a79b420 libsystem_c.dylib`abort   129
libsystem_c.dylib`abort:
    0x7fff9a79b420 < 129>: movl   $0x2710, �i             ; imm = 0x2710
    0x7fff9a79b425 < 134>: callq  0x7fff9a76ce68            ; usleep$NOCANCEL
    0x7fff9a79b42a < 139>: callq  0x7fff9a79b42f            ; __abort

libsystem_c.dylib`__abort:
    0x7fff9a79b42f < 0>:   cmpq   $0x0, 0x8c6dcf1(%rip)     ; gCRAnnotations   7
(lldb)
frame #3: 0x00007fff9a762893 libsystem_c.dylib`__assert_rtn   320
libsystem_c.dylib`basename_r:
    0x7fff9a762893 < 0>: pushq  %rbp
    0x7fff9a762894 < 1>: movq   %rsp, %rbp
    0x7fff9a762897 < 4>: pushq  %r15
    0x7fff9a762899 < 6>: pushq  %r14
(lldb)
frame #4: 0x0000000100656b19 libremote.2.5.4.dylib`boost::intrusive_ptr::operator->(this=0x000070000d1c3160) const   73 at intrusive_ptr.hpp:198
   195
   196      T * operator->() const
   197      {
-> 198          BOOST_ASSERT( px != 0 );
   199          return px;
   200      }
   201
(lldb)
frame #5: 0x00000001007cbd43 libremote.2.5.4.dylib`icinga::CreateObjectHandler::HandleRequest(this=0x0000000101b0c380, user=0x000070000d1c4370, request=0x000000010656d988, response=0x000070000d1c4158, params=0x000070000d1c36a0)   1139 at createobjecthandler.cpp:62
   59       Zone::Ptr localZone = Zone::GetLocalZone();
   60       String localZoneName;
   61
-> 62       if (localZone && !attrs->Contains("zone")) {
   63           localZoneName = localZone->GetName();
   64           attrs->Set("zone", localZoneName);
   65       }
(lldb) p attrs
warning: could not load any Objective-C class information. This will significantly reduce the quality of type information available.
(Ptr) $0 = {
  px = 0x0000000000000000
}

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-12-05 15:38:25 00:00

  • Assigned to changed from magnus to mfriedrich

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-12-05 15:38:29 00:00

  • Status changed from Assigned to Resolved
  • Done % changed from 0 to 100

Applied in changeset 338f5c0.

@icinga-migration
Copy link
Author

Updated by magnus on 2016-12-06 20:15:45 00:00

mfriedrich wrote:

Applied in changeset 338f5c0.

Todays snapshot has ran since this morning without crashes so that is a big improvement.

Thanks

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-12-07 08:24:58 00:00

Yeah sorry, should code more defensive. Was an old patch, normally I would catch such null ptrs.

@icinga-migration icinga-migration added bug Something isn't working area/api REST API labels Jan 17, 2017
@icinga-migration icinga-migration added this to the 2.6.0 milestone Jan 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/api REST API bug Something isn't working
Projects
None yet
Milestone
2.6.0
Development

No branches or pull requests
1 participant
@icinga-migration