What have you done today thread?

[KungFooBob]

Spoken with an ex-colleague.

The organisation has the Akira ransomware virus.

Rather than just encrypt the user data it seems to have totally destroyed the OS on all the hosts and deleted all the local backups. It's even borked the raid config on the backup server.

Not a problem it's all backed up by Veeam to the cloud...

Only they're encrypted backups and the saved encryption key seems to be incorrect.

They're utterly and totally fucked.

[Yorick]

Spoken with an ex-colleague.

The organisation has the Akira ransomware virus.

Rather than just encrypt the user data it seems to have totally destroyed the OS on all the hosts and deleted all the local backups. It's even borked the raid config on the backup server.

Not a problem it's all backed up by Veeam to the cloud...

Only they're encrypted backups and the saved encryption key seems to be incorrect.

They're utterly and totally fucked.

Oops

[Yorick]

I've got me mankini at the ready for when it's refilled and up to temp.

Ooh. Forgot about that. Might take a couple of weeks for sun to warm up.

Bugger. It's only 23c. Not going in till it's high 20s.
It's 30c sun today, sill warm up soon

[Mr. Dazzle]

Spoken with an ex-colleague.

The organisation has the Akira ransomware virus.

Rather than just encrypt the user data it seems to have totally destroyed the OS on all the hosts and deleted all the local backups. It's even borked the raid config on the backup server.

Not a problem it's all backed up by Veeam to the cloud...

Only they're encrypted backups and the saved encryption key seems to be incorrect.

They're utterly and totally fucked.

Bah! There'll be an admin somewhere who's got it all on a caddied hard drive in his desk "just in case".

[KungFooBob]

Spoken with an ex-colleague.

The organisation has the Akira ransomware virus.

Rather than just encrypt the user data it seems to have totally destroyed the OS on all the hosts and deleted all the local backups. It's even borked the raid config on the backup server.

Not a problem it's all backed up by Veeam to the cloud...

Only they're encrypted backups and the saved encryption key seems to be incorrect.

They're utterly and totally fucked.

Bah! There'll be an admin somewhere who's got it all on a caddied hard drive in his desk "just in case".

They only have one IT person. It was me. Then they sacked my replacement after 6 months and made the technical (product expert) guy who used to sit next to me IT Manager because he knew how to reset passwords. They're proper fucked if they can't find that encryption key (that I didn't even know existed, so I'm glad it didn't happen on my shift).

[ZRX61]

They only have one IT person. It was me. Then they sacked my replacement after 6 months and made the technical (product expert) guy who used to sit next to me IT Manager because he knew how to reset passwords. They're proper fucked if they can't find that encryption key (that I didn't even know existed, so I'm glad it didn't happen on my shift).

The correct response is "well, good luck with that, I'm off for a pint"

[ZRX61]

Dropped $6 for tonight's $850,000,000 lottery & $4 for Friday's pittance of a mere $267,000,000.

Bugger, no winners, it rolled over. Looks like I'll have to console myself with winning Saturdays $925,000,000 jackpot.

Double bugger. It ended up being $968,700,000.
The bad news: I didn't win
The good news: Neither did any other bastard... so $1,040,000,000 for Monday.

[Horse]

Yesterday evening.

Another free event in the town. They're always a bit bonkers.

https://www.newburytoday.co.uk/news/don ... e-9332363/

The talented French company Compagnie L’Homme Debout will be gracing the town centre with Mo and the Red Ribbon.

The show tells the story of a child refugee named Mo and his optimistic outlook and fantastical journey towards a new life.

20230930_220731.jpg (906.23 KiB) Viewed 445 times

20230930_220220.jpg (783.48 KiB) Viewed 445 times

20230930_214836.jpg (1.04 MiB) Viewed 445 times

[the_priest]

Life changes for me.

It was announced today that The Bishop of Rochester is pleased to announce that The Revd aka the_priest is to be appointed as the next Priest -In-Charge of the Benefice of St Botolph’s Northfleet and St Marks Rosherville. We look forward to welcoming the_priest and his family to our Parish, hopefully before Christmas.

So big adventures ahead, some good biking roads not too far from there and so much to look forward to in life and faith.

[Yorick]

Spoken with an ex-colleague.

The organisation has the Akira ransomware virus.

Rather than just encrypt the user data it seems to have totally destroyed the OS on all the hosts and deleted all the local backups. It's even borked the raid config on the backup server.

Not a problem it's all backed up by Veeam to the cloud...

Only they're encrypted backups and the saved encryption key seems to be incorrect.

They're utterly and totally fucked.

Bah! There'll be an admin somewhere who's got it all on a caddied hard drive in his desk "just in case".

They only have one IT person. It was me. Then they sacked my replacement after 6 months and made the technical (product expert) guy who used to sit next to me IT Manager because he knew how to reset passwords. They're proper fucked if they can't find that encryption key (that I didn't even know existed, so I'm glad it didn't happen on my shift).

Any update?

[KungFooBob]

Bah! There'll be an admin somewhere who's got it all on a caddied hard drive in his desk "just in case".

They only have one IT person. It was me. Then they sacked my replacement after 6 months and made the technical (product expert) guy who used to sit next to me IT Manager because he knew how to reset passwords. They're proper fucked if they can't find that encryption key (that I didn't even know existed, so I'm glad it didn't happen on my shift).

Any update?

As of 6pm on Friday, they were still fucked.

All four European offices are down. All the data (15tb of it) is there in the cloud backup, they just can't decrypt it. I can't see the business being able to continue trading without the ERP (four separate instances of SAP Business One).

I think the biggest problem was the rush to flatten the existing server estate because they thought they could restore from the cloud. Restores had been tested, but using the on-prem Veeam Enterprise Manager that had the encryption key cached, wasn't until they tried to restore to fresh installed Hosts with the Enterprise manager not available (it was on-prem and wiped) they realised you needed the keys.

It's quite sobering how fucked a company is without it's data and how easily that data can be taken away.

With hindsight, they would have been better off paying the ransom.

[DefTrap]

With hindsight, they would have been better off paying the ransom.

I think paying the ransom is increasingly common. Companies have terrible disaster recovery policies and they don't give them adequate priority. You can't afford to be offline for weeks let alone lose all your data.

[Demannu]

They only have one IT person. It was me. Then they sacked my replacement after 6 months and made the technical (product expert) guy who used to sit next to me IT Manager because he knew how to reset passwords. They're proper fucked if they can't find that encryption key (that I didn't even know existed, so I'm glad it didn't happen on my shift).

Any update?

As of 6pm on Friday, they were still fucked.

All four European offices are down. All the data (15tb of it) is there in the cloud backup, they just can't decrypt it. I can't see the business being able to continue trading without the ERP (four separate instances of SAP Business One).

I think the biggest problem was the rush to flatten the existing server estate because they thought they could restore from the cloud. Restores had been tested, but using the on-prem Veeam Enterprise Manager that had the encryption key cached, wasn't until they tried to restore to fresh installed Hosts with the Enterprise manager not available (it was on-prem and wiped) they realised you needed the keys.

It's quite sobering how fucked a company is without it's data and how easily that data can be taken away.

With hindsight, they would have been better off paying the ransom.

Have they tried turning it off, then back on again?

Up at 7.50 today to watch a bit of water biking.
Then off to a friend who's electric pool cover was/has been a bit recalcitrant for the last couple of years.
Had to re-track 10 channels, then diagnose why it didn't work. For some reason it had 2 limit stop sensors, and of course 1 was fubar. So now waiting on a parcel delivery!

[Mr. Dazzle]

Companies have terrible IT and they don't give it adequate priority.

FTFY.

And I hate IT!

[DefTrap]

FTFY.

And I hate IT!

Yeah but this is the problem, total absence of giving a fvck, let's leave it to IT. Business Continuity is more than just IT getting some backups right.

[the_priest]

I am so glad to have left the world of IT. God stuff is much better, you can pray about it!

We had full redundancy, online backups, off site storage, images and more. That was the one thing that we managed to drive into the CEO's head was that if the data was ever compromised, we'd have a GDPR nightmare on our hands, never mind the issues of lost data. Our lot trained as white hat hackers as well in being able to do penetration testing and to ensure the network was secured and backed up. That is all 5 years ago now and that landscape has probably changed again having moved fully into the cloud.

Sometimes it is just worth investing in things you might never need, because when you do need it, you look brilliantly clever and have saved the day! And keep good staff in place with decent pay and incentives!

[Skub]

I was out on the Zed today and called in with an old mate who's currently in a care home as his wife finds difficulty looking after him 24/7. He's younger than me,but badly disabled by a stroke about 7 years ago. I hate to see the way things have turned out for him,he's a clever bloke,but stuck in there with dementia sufferers and all the usual care home shenanigans. He's fallen twice already and is looking really bashed about.

He's always positive and cheerful and I get a massive reality check every time I speak to him. Sometimes we really don't know how fortunate we are.

[Greenman]

They only have one IT person. It was me. Then they sacked my replacement after 6 months and made the technical (product expert) guy who used to sit next to me IT Manager because he knew how to reset passwords. They're proper fucked if they can't find that encryption key (that I didn't even know existed, so I'm glad it didn't happen on my shift).

Any update?

As of 6pm on Friday, they were still fucked.

All four European offices are down. All the data (15tb of it) is there in the cloud backup, they just can't decrypt it. I can't see the business being able to continue trading without the ERP (four separate instances of SAP Business One).

I think the biggest problem was the rush to flatten the existing server estate because they thought they could restore from the cloud. Restores had been tested, but using the on-prem Veeam Enterprise Manager that had the encryption key cached, wasn't until they tried to restore to fresh installed Hosts with the Enterprise manager not available (it was on-prem and wiped) they realised you needed the keys.

It's quite sobering how fucked a company is without it's data and how easily that data can be taken away.

With hindsight, they would have been better off paying the ransom.

I've been joking around the fact that some companies (and schools) put so much encryption and security processes in place that soon you wont be able to get into your own data/systems, it seems this has come true in your companies case.

A MAT around my way (CSET) had a ransomware attack on a cluster of 28 schools, i think they ended up paying numerous hundreds of thousands of pounds to recover some of it. It also highlighted the fact they had no backups for any of their primary schools. The LA then forced the schools to come with them for support as the MAT's IT dept was forced to close. Ironically they didn't give them this as a choice as they said if they chose to go with a 3'rd party IT support company like the one i work for they wouldn't give them access to the data they had managed to recover, so in theory they ransomed the ransomed data back to the school!

Why wasn't someone monitoring the backups and doing a test recovery to ensure they are working at least once a month at your place?

[KungFooBob]

I've been left 10 months. I did test restores when I was there.

The problem is the testing wasn't good enough. Yes they could restore data using the on-prem enterprise manager because it had the keys cached. I used it for full vms and individual files all the time.

Soon as the enterprise manager is out of the picture you have to use the key you made a note of when you configured the job.

Their noted key doesn't work. Might be a typo, might be the jobs were deleted and new ones created without the owner updating the notes.

Either way, it's not my problem, which I'm unbelievably pleased about.

[Greenman]

I've been left 10 months. I did test restores when I was there.

The problem is the testing wasn't good enough. Yes they could restore data using the on-prem enterprise manager because it had the keys cached. I used it for full vms and individual files all the time.

Soon as the enterprise manager is out of the picture you have to use the key you made a note of when you configured the job.

Their noted key doesn't work. Might be a typo, might be the jobs were deleted and new ones created without the owner updating the notes.

Either way, it's not my problem, which I'm unbelievably pleased about.

I take it who ever created the key and has not got the correct key is in the shit then?

If that would of been me, i would of copied and pasted the key, took a picture of it, sent it to myself on email and saved it on about 3 different secure network drives to make sure i had it safe, too paranoid about those things myself after i had a bit locker issue with a missing or incorrectly written key about a year ago.