Skip to content

fix for Splunk Support Case: 204977#111

Closed
whisperstream wants to merge 1 commit intosplunk:masterfrom
whisperstream:master
Closed

fix for Splunk Support Case: 204977#111
whisperstream wants to merge 1 commit intosplunk:masterfrom
whisperstream:master

Conversation

@whisperstream
Copy link
Copy Markdown

This is a fix for the support Case 204977 which I opened a while back: Without this fix, whenever I tried to execute a oneshot query, I would get back the following exception (this pull request fixes this issue):

File "/usr/local/lib/python2.7/dist-packages/splunklib/client.py", line 305, in connect
return Service(**kwargs).login()
File "/usr/local/lib/python2.7/dist-packages/splunklib/binding.py", line 764, in login
session = XML(body).findtext("./sessionKey")
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
return parser.close()
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err

@itay
Copy link
Copy Markdown
Contributor

itay commented Jul 10, 2015

@whisperstream can you post a small script that shows your problem without your fix? We have many tests and usage of oneshot queries which work fine, so I want to verify.

@whisperstream
Copy link
Copy Markdown
Author

import splunklib.client as SplunkSDKClient
print str(SplunkSDKClient.connect({
'host': ‘
_’,
'port': 443,
'scheme': 'https',
'username': ‘_
’,
'password': ‘**
****’,
}).jobs.oneshot('search * | head 10', *
{'count': 0}))

------------ results without fix ----------------------
ubuntu@spark:$ python test.py
Traceback (most recent call last):
File "test.py", line 7, in
'password': '*********',
File "/usr/local/lib/python2.7/dist-packages/splunklib/client.py", line 305, in connect
return Service(
*kwargs).login()
File "/usr/local/lib/python2.7/dist-packages/splunklib/binding.py", line 764, in login
session = XML(body).findtext("./sessionKey")
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
return parser.close()
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: no element found: line 1, column 0
ubuntu@spark:
$

------------- results with fix ---------------------
ubuntu@spark:~$ python test.py

_bkt _cd _indextime _raw _serial _si _sourcetype _time host index linecount source sourcetype splunk_server main~94~3956FB27-9471-427A-A1AA-06F876745CE1 etc....

@itay
Copy link
Copy Markdown
Contributor

itay commented Jul 11, 2015

Just tried this locally with the SDK, no issues:

import splunklib.client as client

print str(client.connect(**{
    'host': 'localhost',
    'port': 8089,
    'scheme': 'https',
    'username': 'admin',
    'password': 'changeme',
}).jobs.oneshot('search * | head 10', **{'count': 0}))

Output is:

$ PYTHONPATH=~/work/splunk-sdk-python:$PYTHONPATH python test2.py      
<?xml version='1.0' encoding='UTF-8'?>

<results preview='0'/>

The issue I have with the fix is that the connection.close() doesn't actually do it seems like. If you look in the Python code, that socket is actually reference counted, and will remain in effect until "response" object holding is closed. If this weren't true, then no request whatsoever would ever work with the SDK, which is clearly the not the case at the moment.

It's very odd that you're seeing the issue. Can you share a bit about your environment (OS, Python version, etc)?

@whisperstream
Copy link
Copy Markdown
Author

Sure, I installed the sdk using:
pip install splunk-sdk

On both Mac OSX (10.9.5) using python 2.7.5 and Ubuntu (14.04) using
Python 2.7.6 and get the same issue on both.

-Axel

On Fri, Jul 10, 2015 at 5:05 PM, Itay Neeman notifications@github.com
wrote:

Just tried this locally with the SDK, no issues:

import splunklib.client as client

print str(client.connect(*{
'host': 'localhost',
'port': 8089,
'scheme': 'https',
'username': 'admin',
'password': 'changeme',
}).jobs.oneshot('search * | head 10', *
{'count': 0}))

Output is:

$ PYTHONPATH=~/work/splunk-sdk-python:$PYTHONPATH python test2.py

The issue I have with the fix is that the connection.close() doesn't
actually do it seems like. If you look in the Python code, that socket is
actually reference counted, and will remain in effect until "response"
object holding is closed. If this weren't true, then no request whatsoever
would ever work with the SDK, which is clearly the not the case at the
moment.

It's very odd that you're seeing the issue. Can you share a bit about your
environment (OS, Python version, etc)?


Reply to this email directly or view it on GitHub
#111 (comment)
.

@itay
Copy link
Copy Markdown
Contributor

itay commented Jul 15, 2015

@whisperstream we've tried now on multiple machines, including clean VMs, and a couple of Python versions, and simply cannot reproduce this. It's also a section of code that hasn't changed since we launched the SDKs.

I think we're likely going to have to decline the PR, because the solution of not calling connection.close will cause us to leak the connection. That said, we do realize you're hitting this issue, so we want to try and help you there.

I just want to doublecheck: when you set it to port 443 and some host, this is where the splunk management port is running (normally 8089), yes? i.e. not the UI ports.

@whisperstream
Copy link
Copy Markdown
Author

yeah normally 8089, but ops team is running it on 443. Very strange that
you're not seeing it on your end, but understand the data leakage issue (a
concern of mine too....but pragmatism won out). I''ll just continue to
patch until I figure it out.....but thanks for your consideration. Will
post back here if I figure out the root cause.

-Axel

On Wed, Jul 15, 2015 at 2:27 PM, Itay Neeman notifications@github.com
wrote:

@whisperstream https://github.com/whisperstream we've tried now on
multiple machines, including clean VMs, and a couple of Python versions,
and simply cannot reproduce this. It's also a section of code that hasn't
changed since we launched the SDKs.

I think we're likely going to have to decline the PR, because the solution
of not calling connection.close will cause us to leak the connection.
That said, we do realize you're hitting this issue, so we want to try and
help you there.

I just want to doublecheck: when you set it to port 443 and some host,
this is where the splunk management port is running (normally 8089), yes?
i.e. not the UI ports.


Reply to this email directly or view it on GitHub
#111 (comment)
.

@itay itay closed this Nov 20, 2015
mateusz834 pushed a commit that referenced this pull request Apr 15, 2026
* Revamp tool settings

* Make tags non-optional

* Replace unittest.skipTest() with pytest.skip()

* Match typehint for tool.tags with the LangChain StructuredTool

* Remove defaults from ToolSettings

* Fix tests, duplication in README

* Update README
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants