Quantcast

RoundUp stopped creating tickets

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RoundUp stopped creating tickets

Lee Higginbotham

Good day to you all,

First, I'm a complete noob to RoundUp.  I just took over a set of systems that utilize RoundUp version 1.4.18-1.1.  Sometime after 0105 hours on 09 NOV 15, RoundUp stopped creating tickets from exchange.  I have poured through logs over the last week as well as looked at mail flow to no avail.  Here are the details:

Ubuntu 12.04 64bit server
RoundUp version 1.4.18-1.1
Exchange 2013
Server sits in a DMZ and there have been no changes to the firewall rules.

The only thing I can see is there was some patching done on the Exchange server about 5 hours previous to this time frame, but I seem to think that is not the cause, as we wouldn't have received one at 0105 hours the next morning after the patching and reboot of the Exchange server.

There have been some changes to the mail flow as we are migrating away from Exchange and the mail flow has changed somewhat and could be the cause.  I haven't totally ruled it out.

My reason for coming to you fine folks today is more to see what ideas you may have as to a solution with RoundUp.  I have many, many more questions, that I will get into in the coming days once I get the mail to ticket feature working again.

Thank you in advance for any and all assistance.

Sincerely,
Lee Higginbotham
Systems Administrator | Rivera Group | [hidden email]
Office: (812) 246-4055

"We strive to do great things!"

------------------------------------------------------------------------------
Presto, an open source distributed SQL query engine for big data, initially
developed by Facebook, enables you to easily query your data on Hadoop in a
more interactive manner. Teradata is also now providing full enterprise
support for Presto. Download a free open source copy now.
http://pubads.g.doubleclick.net/gampad/clk?id=250295911&iu=/4140
_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RoundUp stopped creating tickets

R. David Murray
On Mon, 16 Nov 2015 15:37:16 -0500, Lee Higginbotham <[hidden email]> wrote:
> My reason for coming to you fine folks today is more to see what ideas you
> may have as to a solution with RoundUp.  I have many, many more questions,
> that I will get into in the coming days once I get the mail to ticket
> feature working again.

Well, you're going to have to look at the inflow of mail into roundup in
order to debug this.  How is mail getting in?  For example, my instance
uses roundup-mailgw POP support to pull the emails in via the POP3
protocol, and I have that set to run out of cron every minute.  How is
yours set up?  If you don't know, look around for *somewhere*, such as
cron or your mail software alias table or, if you are on windows...well,
I don't know on windows...where 'roundup-mailgw' is called.

--David


------------------------------------------------------------------------------
Presto, an open source distributed SQL query engine for big data, initially
developed by Facebook, enables you to easily query your data on Hadoop in a
more interactive manner. Teradata is also now providing full enterprise
support for Presto. Download a free open source copy now.
http://pubads.g.doubleclick.net/gampad/clk?id=250295911&iu=/4140
_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RoundUp stopped creating tickets

Lee Higginbotham

David,

Thank you for your reply.  I too am using POP3 and am using the roundup-mailgw with a cron job to pull everry 5 minutes.

Here are the contents of roundup-mailgw:
 

#! /usr/bin/python
from roundup.scripts.roundup_mailgw import run
import sys, traceback
import datetime
 
 
logFile = '/var/log/roundup/mail_gateway.log'
 
try:
    # raise Exception("I know python!")
    run()
    with open(logFile, "a") as myfile:
        myfile.write('[%s] Mail Gateway Ran\n' %datetime.datetime.now())
 
except SystemExit:
    error = 0
    with open(logFile, "a") as myfile:
        myfile.write('[%s] Mail Gateway Ran\n' %datetime.datetime.now())
 
    #print 'Nothing to see here, normal exit'
except:
    print 'Gateway Error: ', sys.exc_info()[0]
    traceback.print_exc(file=sys.stdout)
    with open(logFile, "a") as myfile:
        myfile.write('[%s] Mail Gateway Error: %s  \n'  % (datetime.datetime.now(), sys.exc_info()[0]) )

This file has an time date stamp of Aug 20 2013, so it hasn't changed in a long time.  I do get several Mail Gateway Errors, but looking back at the historical logs, we have been receiving these errors for quite some time and that doesn't appear to be the issue.

Sincerely,
Lee


 

Quoting "R. David Murray" <[hidden email]>:

On Mon, 16 Nov 2015 15:37:16 -0500, Lee Higginbotham <[hidden email]> wrote:

My reason for coming to you fine folks today is more to see what ideas you
may have as to a solution with RoundUp.  I have many, many more questions,
that I will get into in the coming days once I get the mail to ticket
feature working again.

Well, you're going to have to look at the inflow of mail into roundup in
order to debug this.  How is mail getting in?  For example, my instance
uses roundup-mailgw POP support to pull the emails in via the POP3
protocol, and I have that set to run out of cron every minute.  How is
yours set up?  If you don't know, look around for *somewhere*, such as
cron or your mail software alias table or, if you are on windows...well,
I don't know on windows...where 'roundup-mailgw' is called.
--David



Sincerely,
Lee Higginbotham
Systems Administrator | Rivera Group | [hidden email]
Office: (812) 246-4055

"We strive to do great things!"

------------------------------------------------------------------------------

_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RoundUp stopped creating tickets

R. David Murray
On Tue, 17 Nov 2015 07:48:22 -0500, Lee Higginbotham wrote:
> Thank you for your reply.  I too am using POP3 and am using the
> roundup-mailgw with a cron job to pull everry 5 minutes.

[...]

> This file has an time date stamp of Aug 20 2013, so it hasn't changed in a
> long time.  I do get several Mail Gateway Errors, but looking back at the
> historical logs, we have been receiving these errors for quite some time
> and that doesn't appear to be the issue.

Yes, roundup-mailgw is part of the roundup software, and ideally you
wouldn't make any local changes to it.

You are on an older version of roundup than I'm using, so I don't
know that I can give you much help.  Basically, you want to correlate
the entries in the roundup mail log with the submissions from your
server that are failing, and see if you can figure out where things
are going wrong.  If roundup is processing the messages but not
updating the issue, and there's no corresponding error in the
mail gateway log, you may need to start adding some logging to the
roundup code to track it down.  But I'm guessing you'll find
something before you get to that point.

People with experience with your version of roundup may have additional
suggestions, once you complete the initial investigation.

--David


------------------------------------------------------------------------------

_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RoundUp stopped creating tickets

Lee Higginbotham

David,

Thank you for your assistance.  I've been beating my head against this issue since the 9th of November.  I've confirmed that all traffic is flowing the way it should.  The only problem seems to be the roundup-mailgw is not working properly all the time.  The only thing in my mail_gateway.log are all the notices that the "Mail Gateway Ran" with an occasional error of this nature:

[2015-11-09 09:01:01.828927] Mail Gateway Error: <class 'ssl.SSLError'>
[2015-11-09 06:25:10.279796] Mail Gateway Error: <class 'poplib.error_proto'>
[2015-11-09 05:56:01.444671] Mail Gateway Error: <class 'poplib.error_proto'>
[2015-11-09 06:01:01.798850] Mail Gateway Error: <class 'ssl.SSLError'>
[2015-11-09 05:26:02.084608] Mail Gateway Error: <class 'poplib.error_proto'>
[2015-11-08 19:06:02.093787] Mail Gateway Error: <class 'ssl.SSLError'>

An coincidentally, since the errors on the 9th, I have received no Mail Gateway Errors at all.  However when the roundup-mailgw command in run, I receive the following in the roundup.log:
 

2015-11-17 22:55:01,913 ERROR POP server error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/roundup/mailgw.py", line 1330, in _do_pop
    server = klass(server)
  File "/usr/lib/python2.7/poplib.py", line 360, in __init__
    self.sslobj = ssl.wrap_socket(self.sock, self.keyfile, self.certfile)
  File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 143, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
    self._sslobj.do_handshake()
error: [Errno 104] Connection reset by peer

I have checked all the certificates and they are all valid and have no issues that I can tell, however, every 5 minutes, I receive these messages.

Again, I inherited these systems and am learning about roundup, so, thank you again for your assistance.  Hopefully, someone else will chime in and provide assistance.

Sincerely,
Lee





Quoting "R. David Murray" <[hidden email]>:

On Tue, 17 Nov 2015 07:48:22 -0500, Lee Higginbotham wrote:

Thank you for your reply.  I too am using POP3 and am using the
roundup-mailgw with a cron job to pull everry 5 minutes.

[...]

This file has an time date stamp of Aug 20 2013, so it hasn't changed in a
long time.  I do get several Mail Gateway Errors, but looking back at the
historical logs, we have been receiving these errors for quite some time
and that doesn't appear to be the issue.

Yes, roundup-mailgw is part of the roundup software, and ideally you
wouldn't make any local changes to it.

You are on an older version of roundup than I'm using, so I don't
know that I can give you much help.  Basically, you want to correlate
the entries in the roundup mail log with the submissions from your
server that are failing, and see if you can figure out where things
are going wrong.  If roundup is processing the messages but not
updating the issue, and there's no corresponding error in the
mail gateway log, you may need to start adding some logging to the
roundup code to track it down.  But I'm guessing you'll find
something before you get to that point.

People with experience with your version of roundup may have additional
suggestions, once you complete the initial investigation.
--David



Sincerely,
Lee Higginbotham
Systems Administrator | Rivera Group | [hidden email]
Office: (812) 246-4055

"We strive to do great things!"

------------------------------------------------------------------------------

_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RoundUp stopped creating tickets

R. David Murray
On Tue, 17 Nov 2015 13:00:38 -0500, Lee Higginbotham <[hidden email]> wrote:

>   An coincidentally, since the errors on the 9th, I have received no Mail
> Gateway Errors at all.  However when the roundup-mailgw command in run, I
> receive the following in the roundup.log:
>  
>   2015-11-17 22:55:01,913 ERROR POP server error
>   Traceback (most recent call last):
>     File "/usr/local/lib/python2.7/dist-packages/roundup/mailgw.py", line
> 1330, in _do_pop
>       server = klass(server)
>     File "/usr/lib/python2.7/poplib.py", line 360, in __init__
>       self.sslobj = ssl.wrap_socket(self.sock, self.keyfile,
> self.certfile)
>     File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
>       ciphers=ciphers)
>     File "/usr/lib/python2.7/ssl.py", line 143, in __init__
>       self.do_handshake()
>     File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
>       self._sslobj.do_handshake()
>   error: [Errno 104] Connection reset by peer
>
> I have checked all the certificates and they are all valid and have no
> issues that I can tell, however, every 5 minutes, I receive these messages.
>
> Again, I inherited these systems and am learning about roundup, so, thank
> you again for your assistance.  Hopefully, someone else will chime in and
> provide assistance.
Well, that would be your answer then.  The POP connection is failing,
so no email gets ingested into roundup.

Did you upgrade python?  Or your exchange server?  SSL error checking
was made more strict in 2.7.9, and again in 2.7.10, although in that case
I'd expect to see an error other than a connection reset.  However,
it may be a cipher problem: either python or your mail server may no
longer accept certain ciphers, and they may not be finding any in common
(though that would be unexpected).

You might want to try writing a small python program using poplib that
tries to do the POP connection to your mail server, and use that to
debug the connection problem.

Also, since the message is that the connection was reset by the peer,
hopefully there's something in the Exchange log that shows *why* it
disconnected the client.

--David


------------------------------------------------------------------------------

_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: RoundUp stopped creating tickets

Lee Higginbotham

David,

Thank you.  

I updated Python on 16 OCT but it was a security release from 2.7.3-0ubuntu3.6 to 2.7.3-0ubuntu3.8.  I also know that the night before all this happened our Windows Admin did push windows updates to the Exchange server, however, mail is flowing normally everywhere.

The only problem is when roundup goes and tries to get the mail from exchange.  I will see about having my Windows Admin look into the logs and we can go from there.

Sincerely,
Lee


Quoting "R. David Murray" <[hidden email]>:

On Tue, 17 Nov 2015 13:00:38 -0500, Lee Higginbotham <[hidden email]> wrote:

An coincidentally, since the errors on the 9th, I have received no Mail
Gateway Errors at all.  However when the roundup-mailgw command in run, I
receive the following in the roundup.log:
 
  2015-11-17 22:55:01,913 ERROR POP server error
  Traceback (most recent call last):
    File "/usr/local/lib/python2.7/dist-packages/roundup/mailgw.py", line
1330, in _do_pop
      server = klass(server)
    File "/usr/lib/python2.7/poplib.py", line 360, in __init__
      self.sslobj = ssl.wrap_socket(self.sock, self.keyfile,
self.certfile)
    File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
      ciphers=ciphers)
    File "/usr/lib/python2.7/ssl.py", line 143, in __init__
      self.do_handshake()
    File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
      self._sslobj.do_handshake()
  error: [Errno 104] Connection reset by peer

I have checked all the certificates and they are all valid and have no
issues that I can tell, however, every 5 minutes, I receive these messages.

Again, I inherited these systems and am learning about roundup, so, thank
you again for your assistance.  Hopefully, someone else will chime in and
provide assistance.

Well, that would be your answer then.  The POP connection is failing,
so no email gets ingested into roundup.

Did you upgrade python?  Or your exchange server?  SSL error checking
was made more strict in 2.7.9, and again in 2.7.10, although in that case
I'd expect to see an error other than a connection reset.  However,
it may be a cipher problem: either python or your mail server may no
longer accept certain ciphers, and they may not be finding any in common
(though that would be unexpected).

You might want to try writing a small python program using poplib that
tries to do the POP connection to your mail server, and use that to
debug the connection problem.

Also, since the message is that the connection was reset by the peer,
hopefully there's something in the Exchange log that shows *why* it
disconnected the client.
--David



Sincerely,
Lee Higginbotham
Systems Administrator | Rivera Group | [hidden email]
Office: (812) 246-4055

"We strive to do great things!"

------------------------------------------------------------------------------

_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Loading...