ESET: "SOME COMPUTER SYSTEMS" with "PERMISSION PROBLEMS"? Really?

I am sympathetic to "sh!t happens", and am not mad about the actual 3435 problem. I am however VERY displeased with your response to this.

That update cause MAJOR outages, and you are referring to it as "SOME COMPUTER SYSTEMS" with a "PERMISSION PROBLEMS"!!!! BS!! How many man hours were spent with people just sitting in front of a toaster?

ESET, you handled this VERY poorly. And unless you can at least recognize this problem for what it was, you will be without a LOT of renewals. Is it too much to ask for you to these questions like you are addressing a bunch of network engineers?

1) How did this problem happen?

2) What measures will you be taking to be sure it wont happen again?

3) Why did it take so long for you to acknowledge the problem on your website?


I am the one who put my ass on the line saying that ESET is a good company, and they have a good product. I am the one who had to look the CEO in the eye yesterday, and explain that things were down because this good company rolled out a bad definition, and cost our company thousands of dollars in wasted labor. The LEAST you can do is pay this issue a little respect and not blow it off like it was a typo!!

 

I agree 100%. The fact that they haven't said anything leads me to believe that the don't know how it happened or the just don't care. I may be wrong with those assumptions, but without any word from them, that's how I feel.

 

What do you want them to do, they took down the definitions, and sent one ones quickly to fix the issue, they made a post acknowledging the issue, and saying it was fixed

What do you want? them to come to your house and give you a big old hug?, issues happen with software, as with everything else in life

While eset may not like me saying this, If your not happy change your antivirus, simple as that, that or wait out your subscription period and change, take your pick, but complaining on a public forum isn't going to get you anywhere

If your talking with a CEO i'm sure you know the CORRECT steps to getting issues resolved, public forum posting is about the last thing in those 'correct' steps

-Brian

Oh and if i may add

for your question 3, probably because FIXING it takes precedent over letting people know, if you can fix it before anyone else has an issue, thats #1 priority

And i still don't understand what you want them to say, how they input definitions and the exact cause of this is most probably vital information to how nod32 works internally, You want them to give you a rundown of how exactly their software works?

@norky, since it's fixed they obviously know what was wrong after a little research

 

Hmm...I think this is a Official ESET Support Forum...YET!...or I'm wrong?

 

Brian,

I think part of the OP's angst is at the way ESET appear to be trying to downplay this on their web site..

ESET report "Virus signature update 3435 caused some Windows systems to report permission errors."

The many forum reports relating to frozen and disabled machines suggest otherwise, unless misunderstanding ESET's quote... I can certainly assure you that my Windows session did not politely report that there was a permission error!! He is also undoubtedly right in that there has been substantial lost productivity from this.

If this was a one off, the more likely that goodwill would automatically prevail; but it's not the first time recently, hence the doubts...

Peter

 

An issue of the magnitude he's speaking of, IE thousands of dollars of loss, needs to be dealt with directly, if he just wants to piss and moan, then yes i'd say he's in the correct place

@pbw3, What you gotta realize though is people effected by this was most probably a small amount, You mention the many threads about this issue, but take into account, most people are complacent until an issue arises, and ofcourse take into account the amount of people that eset has as customers, and the amount of people complaining, it seems small(ofcourse thats just 1 of many many variables to understand the relationship for this problem), ofcourse these 0 chance eset will release the exact number of people effected(as it's very hard to pinpoint, and it'd be a dumb business move), i'd still venture a guess that it's a small % of people

Yes it sucks if you were one effected, But sadly thats life

-Brian

If you sit back and look at everything, and the amount of various computer configurations there are and could be, and the fact that you attempting to make your software work on every single one of those without issues, it's just not going to happen, But perhaps since i try to understand the issues in making software a bit more, i see things differently, i dunno

 

Actually there was a bug in v. 3.0.650/657 that was fixed shortly after their release using an automatic module update. Unfortunately, the fix made was circumvented by the update 3435 for certain technical reasons in error which made the bug manifest mostly on win2k systems. We have found the cause reasonably quickly, stopped the updates as soon as we became aware of the issue and released a new one with a fix after 3-4 hours after the release of the problematic update. I must say that neither we nor any of our distributors were able to reproduce the issue. I can assure you that updates are tested thoroughly before they are released, however, we cannot run it on all system configurations. If one would be able to do that, there would be no service packs nor hotfixes for Windows either as it would be 100% perfect from the instant of the release. I can assure you that we are doing our best to minimize the number of issues and we have taken measures to minimize the impact in the event of a failure. We hope that such an issue will not occur any more in the future and we'll do our best to ensure the quality you have been used to. One of the measures we have taken is watching this forum carefully after each update and react immediately in case of reported update issues. Should you come across any issue, please always contact customer care as well besides posting it here.

 

@Kayracc:
That's why the problems should be adressed here...not only because a need for "big old hug".

 

The OP obviously has a friend and or client that has a major business operation, A post here mentioning the problem is a good idea

a post here after the fact that it's been fixed mentioning his 'thousands of dollars' of lost man hours and demanding they freak out about the issue, is a little bit far fetched from what marco's noted btw

-Brian

but again, just my view

 

Thank you very much Marcos, it really does make me feel much better knowing exactly which builds and definitions were affected. The fact that you are willing to address this means a lot to me as well. I think that when there is a bug that causes problems like this, you should always post all of the effected versions at least in the forums. It allows people to identify a problem much quicker.

In the future, it would be VERY helpfull to have something like an official sticky thread, and if the problem warrants, even a link to the thread from the support section website. When your network is critical, you do not always want to dig through a thread to see if there is another user with a "maybe" fix. I know there is no perfect solution, but those of us who were having the problem needed more than what was available.

Here is how I think ESET should have handled this. As soon as you found the cause, and pulled the update, you should have posted that to a thread, put it on the top and made it red with flashing lights. The subject should have been "Problem with 3435 update". The body should have been just what you told me, or what you knew at the time. "We have pulled 3435 update, there are sporadic problems with versions 3.0.650/657. Workaround is to disable real time scanning option. Attached is a NOD32 config file that will disable the problem when pushed out with RA" or whatever the name of that executable check box is. That would have saved my network at least an hour and a half of testing, rolling updates, checking, and would have made me confident the problem was not getting worse.

Really that post would have made my world SO much better.

Once again, thanks for addressing this.

@Kayracc:
THis is going to be a long post, and I hope that it is valuable to ESET as well as you to help see my point. I thank you for being devils advocate and allowing me to address this problem to the extent I would like to.

I don't demand that anyone freak out, and I am brought up the money because this was a VERY VERY serous issue for not only me, but a LOT of people who were posting on the primary problem thread, and from ESET's response, I felt like it was not paid the attention it deserved. I hope for everyone's sake that it was only a small percentage of people with this problem, but the fact remains that I had the problem, and after reading the problem thread, there were a lot of people whit the problem. As of right now there are 2800 view,and 63 posts. Of those view, how many of those were network admins with more than one license? Whether I am just pissing and moaning, I think that ESET would happy to know how serious this problem is to me and would like the opportunity to address my concerns, and possibly keep my renewals. I thought I did a good job of keeping my points clear, and free of emotion. If you think that the money was an exaggeration, do the math. If the PCs were hosed for 2.5 hours at mean employee wage of lets say $17/hr, that means that for every 23.5 employees we had, we lost $1,000. Not to mention bad pr telling customers who call that we cant see their bills, and they will need to call back, and what about the overtime that some employees will need to work to get their work done? Thousands of dollars was not an exaggeration for my company by any means, and after reading the problem forum, it looks like maybe not an exaggeration for a lot of companies. I am not looking my money back or anything like that. I am just trying to get my point across that this was a serious problem for me, and some other users. And is this not "dealing with it directly"? What other choice for dealing with it do I have? I could call, and one person will hear what I have to say? I surely don't want to sue, nor do I have the grounds to sue. I guess i could go to the BBB, and tell them my AV company made a mistake? I think this is my best option to address my concerns with ESET.

As far as being to busy to address while they are fixing it, I personally disagree. In my experience, when I crash a router that has 4,500 users on it, it is better to delay the fix one minute and let people know what is going on. I have done it both ways, and by FAR the preferred way is to tell them what is happening. If people don't know what is happening, they will keep calling and troubleshooting, often time causing additional problems, and so on. If I let people know, "hey the problem is our fault, it will be up in 10 minutes, or 2 hours" they may not be happy that it is down, but the will not waste time trying, and will either find a work around, or find something else to do. Granted, that is an ISP router, and not a AV definition update, maybe circumstances are different.

I can only assume that you did not have any machines affected by the problem, let alone, 10, 50 or hundreds machines all feeding from a local update mirror. For those of us who did, the problem instills a lot of fear. I live in a controlled network where everything is tested before it is rolled out to clients. The reason that we test is because the proper functioning of computers is necessary to conduct business. A lot of effort and money is spent making sure that if there is a problem it is localized. We have NOD32 because it does an excellent job protecting machines, and seems to be a real professional product. When we do our security audits every year, I sit down with some colleagues, and we try to think about any possible way that the network could be disturbed, whether in part or in full. From what we come up with, we then make proposals to upper management about what equipment or software we need to purchase help the network uptime. Management doesn't understand what we are asking for or why we are asking for it, but they trust us that when we say we need to spend $30,000 with cisco to make sure that our switching infrastructure is solid, the assume that the $30k they just spent is them doing their part to assure that the network will be stable until next year. You and I know that is not possible, but that is joy of dealing with a non-technical management. I have seen antivirus cause problems before, and I get that. I explain constantly that security by nature comes at an inverse of convenience, whether it is because the spam filter had a false positive, or an antivirus definition update cripples Office.

This was different. This was something that we never considered. While I knew it was technically possible, It was written off as the odds being infinitesimally small. What are the odds that an antivirus will false positive EVERYTHING. Well, it just did, and it did it automatically. In the matter of 5 minutes, pretty much every computer on the network was either dead, or a ticking time bomb, soon to be dead. The computers couldn't even run CMD.

"no big deal, just disable it until the definitions are updated". Well, we didn't know it was the definitions, hell we didn't even know it was NOD32 right away. All that took time. Once we knew it was nod32, what was it, was it the definition update, was it the build, blah blah. We didn't know. We saw that the new updates did not seem to be having the problems, but then again, not everyone with the same version that we were having the problem with had the same problem. What the hell was it? Was the the update a hotfix, was it a permanent bug fix, was once we got the update in was it safe to turn the scanning engine back on? What do we tell management, is ESET aware of it, are the working on it?

Coming to the realization that a simple definition update that can happen automatically twice a day can COMPLETELY *&#@ my word, I really don't think that (while a little late) it is too much to ask to see how ESET works internally when something like this pops up. Are they a huge corporate machine, or do the developers them selves scour these forums. If these questions were answered in the FAQ, I wouldn't be asking here.

So no, I don't want a "big old hug". I don't even want to change antivirus software, because I really like NOD32. I don't want to see the patched C code that fixed the bug. I want to know what the company policy is. I want to see a response from some one that that means something, and not a corporate blow off like I would expect from Symantec. And as I pointed out, and then as you pointed out again, yes, if they are not equipped to do that then I may not renew, or I may have to change early. Maybe I have the wrong AV for my needs, maybe there is no great solution. I don't understand how a definition update can block access to everything on the system, nor do I really want to know. I just want to be REASSURED.

So that was a lot of words to help you see my point of view, and hopefully it will hope it will get some talk going inside ESET to make a good product better. I personally hope that I never have to change AV.

 

cody, great post! you've put to words everything I felt. thank you! I owe you a beer.

 

This is ESETs "Official Support Forums....frequented by Eset staff/personnel. Take a peek at the root of the forums..that head that says "Official ESET Support Forum".

Esets customer relation staff would be different from the programmers hunched over their keyboards and monitors feverishly trying to fix the bad code. Public relations/Eset reseller representatives taking the time to communicate with the many Eset resellers that frequent this site..would not take away time from the Eset programmers trying to fix the issue. It's not like it's a 3 man show over there. (usually)

A lot of us have to vent, and have the right ot vent. A lot of us are IT consultants here...who are Eset resellers. We've installed Esets enterprise edition products on many..many..many networks..that are clients of ours. When this happens...when a product that we recommend to our clients has a major issue like this....it is egg on our face, we're expected to fix it..in a hurry...for free. Free because it's a product we pushed on our clients. What is a consultant to do, when on a day of a bad release like this....several dozen clients call your cell phone in a panic...each of these several dozen clients are networks from say..10..to 300 PCs in size. Spread across several states. Carefully picture this scenario for a minute...and figure out how to help all of these people at once..so that they can get back into productivity again. Clients are losing productivity, because their network is down. That's money to them. We..as consultants, are losing money...because we're "volunteering" our time in fixing this major issue on a product we pushed on the client, instead of going out doing normal productive consulting work at a 100 or 150 or more bucks an hour. So we blow at least one day of billing..that's a lot of money NOT getting in our pocket.

Luckily I'm still wary about version 3..I only have 1x small client of 20x PCs running on version 3, allllllll my others are still on trusty 2.7.

The yearly Exchange server hiccups they had were tough enough, but a major one like this which impacts workstations..that's tough to bounce back from.

 

A few years ago, an anti-spam product we were using took a SPAM definition update that flagged every single e-mail as SPAM. This stuff happens. But like a lot of others here, I'm bothered more by the way things were handled than the actual problem.

What I will never understand is why a company like ESET, which requires an e-mail registration (at least for the corporate version), wouldn't immediately send an e-mail to all customers notifiying us of this problem. Maybe I'm missing something, but that seems like such an easy task and it would go a long way towards allowing those of us affected by an issue to mitigate the damage.

Of course, if VMWare can effectively disable most ESX functionality with a major screw-up, and likewise not bother to notify anyone as soon as they find out, perhaps I'm expecting too much of the industry these days.