What Facebook Knows About You


fb

There’s yet more hype this week around Facebook and privacy, coming out of the release of a new feature ‘Off-Facebook Activity’, which is now available in some regions, Ireland being one of them. This new feature allows you to view (and clear) activity from non-Facebook entities. So, this is basically information about third-party websites or applications that share your visit history with Facebook.

For example, you visit the Harvey Norman website and buy a laptop. Harvey Norman shares this information with Facebook, and the next time you visit Facebook you see an advertisement for a laptop bag. This is one of the main ways that Facebook will use to target advertising. Now by going to Settings -> Your Facebook Information -> Off-Facebook Activity you can see each site that has shared information with Facebook in this way. Most normal users aren’t even aware that this is happening, and that sites they visit completely independently of Facebook will drive the ads they see on the platform.

When I checked this out of my own profile, I was not surprised to see that 152 apps and websites had shared information about my browsing habits with Facebook. The most recent activity was from Microsoft, where I had recently been looking to buy a new Surface Pro on Microsoft.com:


ms

This is a step in the right direction in terms of transparency of this behavior, and I like the fact that I can now remove this data if I chose to also. But what else does Facebook know about me?

For a while now, Facebook has provided the ability to request a download of all of the information that it stores about you as a user of the platform. All you need to do is request it, and about an hour or so later you’ll receive a link to download a compressed (ZIP) file that contains a treasure trove of your personal information.

To generate your download:

  • Go to Settings
  • Go to Your Facebook Information
  • Go to Download Your Information
  • Under Request Copy, select Create File

I decided to give this a try to see exactly what information Facebook has collected from my 12 years of being an active user. The file itself can be large, mine was around 500MB. But what exactly does Facebook store about me? It intrigued me to think that all this data is sitting in some Facebook data center, so I wanted to know exactly what was there. Let’s delve into the download and see exactly the type of information that Facebook has stored on me long term.

The structure of the downloaded file looks something like the below, containing a bunch of folders each containing information relating to specific areas:


fb-download

I spent a while digging through the information. There are quite a few areas that concerned me. Firstly, the ‘ads’ folder. This contained three files:

  • ads_interests – a large list of what Facebook perceives my ad interests to be.
  • advertisers_who_uploaded_a_contact_list_with_your_information – a list of advertisers who uploaded a list to Facebook with my email address.
  • advertisers_you’ve_interacted_with – a list of every ad I’ve ever clicked on within Facebook.

The information stored here is very valuable to Facebook in terms of its advertising business – for example, let’s say I clicked on a craft beer ad (which I often do), and a new craft beer business wants to target relevant users in my region, then I would be highly likely to be in that list of targeted users based on the information that Facebook has on me. This rudimentary approach to targeted advertising contributed to Facebook surpassing $16 billion in advertising revenue as of the end of 2018.

What else do we have in the download? Digging further, I discovered that the following information was present:

  • Every event from Facebook that I have ever been invited to, attended or setup.
  • My friends list along with all the friend requests I have ever made or rejected.
  • A list of all the groups I’ve ever joined.
  • Every page and comment I have ever liked on Facebook.
  • Every messenger thread I have ever been involved in, with all the private conversation content.
  • Everything I’ve ever posted to my Facebook profile.
  • Within the ‘about_you’ folder, I found a file called ‘your_address_books’ which contained all the contacts and phone numbers from my iPhone – this was alarming as I never remember allowing any application or Facebook access to this data.
  • All photos and videos including all my photo album content came in the download (this explains the large size).

My ‘location’ folder was empty, as I had disabled location tracking on Facebook long ago, but if you didn’t this folder would contain a list of the locations (including GPS coordinates) where you have ever logged on to Facebook.

What’s the bottom line here? Facebook stores a crap load of data about you and uses it to drive its advertising business. Like it or not, that’s the truth. If someone had access to the ZIP file that I downloaded, they could likely build a complete profile on me, see all my previous private conversations with friends, access friends phone numbers, see ads that I clicked on, and also determine sites that I have visited separately from Facebook.

There are a few things you can do to ensure that you lock down your advertising settings, which I recommend that you do:

  • Clear your Off-Facebook Activity regularly.
  • Turn off Location History.
  • In Ad Settings, set ‘Ads based on data from partners’ to ‘Not Allowed’.
  • In Ad Settings, set ‘Ads based on your activity on Facebook Company Products that you see elsewhere’ to ‘Not Allowed’.
  • In Ad Settings, set ‘Ads that include your social actions’ to ‘No One’.

These can help, but ultimately Facebook is constantly updating a profile on you based on your browsing activity. We all take Facebook usage at face value, but we forget that at the end of the day, Facebook is a business and is using all of our personal data to drive one of its main revenue sources – advertising.

I am reminded of my favorite comedian Bill Hicks’ thoughts on advertising.

Transitioning from an Individual Contributor to a Leader

I made a transition from an individual contributor to a leadership role 6 years ago, in February 2013. If you’ve made this transition in the software development world, you probably know that it can be difficult and there can be a few things that people may struggle with at the beginning. If you are thinking about moving to a leadership role, or have recently moved, this post may help. Here I’ll share some of my thoughts and some of the advice that helped me successfully make the transition.

An element of my new role I recall struggling with very early-on was the feeling that I was no longer making a tangible contribution to a software development project i.e. committing code to Git or testing other developers code. It took me a while to realize that I needed to now begin focusing on the project delivery as a whole and ensuring that was successful, and not necessarily get down into the weeds unless I really had to. The other item a new manager needs to be able to do early-on is to trust his or her team, and some may struggle with this, especially if they have come from a technical lead position in the team. This can manifest in a dictatorial style of management, which is not good for the team, or the business, and ultimately will not end in a successful outcome.

I’d like to share a few key pieces of advice I was given early on in my career, that have helped me make the transition from individual contributor.

“Be a leader, not a manager.”

This is rather cliché, and if you have ever attended any management training over the years, or read certain authors, you’ve probably heard this a lot. In my opinion, a manager is a strictly human-resources centric term – they handle general people management items like holding 1-1 meetings, performance reviews, they track leave balances, and perhaps annoy developers when they are 5 minutes late back from lunch. I picture someone with a clip-board and a pen every time I hear the word ‘manager’.

A leader is engaged day-to-day on the projects their team is working on. They know what is going on in each one, not necessarily every line of code, but they are familiar with each feature and the status. They are technical and can weigh in on technical discussions if required (although they generally don’t, as they trust their team’s ability to make the right decisions, and learn from their mistakes when they don’t). They have a continual eye on quality – quality of ongoing projects, and quality processes and how they can be improved. Leaders care about people’s development and ensuring that their team are working towards their ultimate career goals (for employees who have them, for those who don’t, that’s OK too). Leaders build a strong working relationship with their direct reports. Leaders are also looking to future – what are the technologies their team should be investing in? What opportunities are we not exploiting? How can we do what we do better?

“If your team is successful, you will be successful.”

This is a simple piece of advice, yet very, very powerful. As an individual contributor, you can ensure that your assigned work is completed to the highest level of quality, but the project may still fail due to another area not being implemented successfully, or some external dependency not being fulfilled. As a leader, you need to ensure you have oversight in all areas that may impact or impede your team’s progress, and be actively working to ensure that any unknowns are clarified, any blockers are removed, any external issues are resolved quickly, and ensuring your team members are focused on what they do best. At times, you will also need to protect your team from external distractions (e.g. getting pulled into unplanned work owned by another team). In my experience, your team will be in a better position to execute successfully if you are actively looking at these items (daily), and by extension you will be successful.

“Your network is important.”

I’ve been told this many times over the years, and only realized the importance of it a few years ago when I wanted to transition to a new role. It’s easy to neglect this, especially if you are new to a leadership role, but I would stress the importance of growing your network with relevant contacts, as you never know what opportunities those contacts may present in the future, or what synergies you can create between for example, different teams based in the same location.

Remember

There will be hard times, and there will always be challenges.

As a software development leader, people will look to you for answers that you don’t always have. You will need to assimilate and recall large amounts of information. You will need to be able to account for your teams time when needed. You will need to explain your failures. You will always need to champion the team’s achievements, stepping back and giving your team the credit that they deserve. You will need to continually learn about new technologies and understand them to enable you to take part in technical discussions. You will need to manage demanding executives, other leaders, stakeholders and customers, and always protect your team in the process.

In a challenging time, I always remember the Winston Churchill quote:

If You’re Going Through Hell, Keep Going.

One of my favorite TED talks by Richard St. John in which he outlines his thoughts on the secrets of success, captures the essence of day-to-day leadership excellently I think – you basically have to persist through CRAP – Criticism, Rejection, Assholes, and Pressure.

The key for me to any leadership position is ensuring that you are continually learning. Remember that in any situation, good or bad, you can learn – especially from the bad situations.

Thoughts on AWS re:Invent 2018



I’ve just returned from AWS re:Invent 2018, Amazon Web Services’ yearly conference showcasing new services, features, and improvements to the AWS cloud. This was the 7th year of re:Invent, and my first time attending.

The scale of the conference is staggering – held across six different Las Vegas hotels over five days, with almost 60,000 attendees this year. I expected queues, and got them. Overall though logistically the conference was well organized. Pending I queued at least 30 minutes beforehand, I was able to to make it to 95% of the sessions I planned on attending across the week.

In terms of the sessions themselves, most were very good. Over the week, I attended sixteen different sessions, made up of talks, demos, chalk talks, and hands-on sessions.

Two of my favorite sessions were ‘Optimizing Costs as you Scale on AWS’ and ‘AIOps: Steps Towards Autonomous Operations’. The former described the 5 pillars of cost optimization – Right sizing, Increasing Elasticity, Picking the Right Pricing Model, Matching Usage to Storage Class, and Measuring and Monitoring. These may seem obvious, but can often be forgotten in instances where the project is a POC that becomes production for example, or a team is not too familiar with AWS and how costs can increase as you scale up an applications usage in production. This session also included insights from an AWS customer who talked through how they had applied and governed this model in their organization, which was interesting to compare and contrast to how I’ve seen it done in the past.

I also attended numerous sessions on SageMaker, AWS’s managed machine learning service (think AML on steroids). I’m looking forward to starting to play around with SageMaker, now that I have attended a hands-on lab I am more confident beginning to look at some of the ideas I have where this could be applied. I looked at this earlier this year while completing my Masters Thesis, but ended up using Amazon Machine Learning instead in the interest of time (AML is a lot simpler to get up and running). AWS also announced Amazon SageMaker Ground Truth, which can be used to streamline the labeling process for machine learning models, via human labelling and automated labelling. One other cool announcement around ML was the launch of AWS Marketplace for Machine Learning, where you can browse 150+ pre-created algorithms and models that can be deployed directly to SageMaker. Someone may have already solved your problem!

If I was to retrospectively give myself some advice for attending re:Invent, it would be:

  1. Try to organize session by hotel. Moving hotels between sessions can take a long time (especially at some points of the day due to Las Vegas traffic). Organizing your sessions so that you are in the same hotel for most of the day can be beneficial. A good thing though is that there is a regular shuttle between conference venues.
  2. Don’t assume you will make every session. Colleagues who had previously been to re:Invent gave me this advice, but I still assumed I would make everything. Traffic, queues or something else will inevitably disrupt your schedule at some point during the week.
  3. Leave time for lunch! Easy to forget when you’ve got a menu of exciting talks to attend. AWS provided a grab-n-go lunch option which was very handy to just grab something between sessions.

If I had one criticism of re:Invent, it would be that some of the talks labelled as advanced did not go as deep as I expected into the technical detail. I thought the hands-on labs did a good job of this though, especially the two I attended on AWS SageMaker.

Overall, re:Invent is a significant investment in the attendees you send (tickets are not cheap, not to mind accommodation, food etc. – remember it’s held in Vegas), but a good idea if you are taking first steps with AWS, looking at getting in deeper or optimizing your usage, or thinking about migrating existing on-premise services to the public cloud.

See here for a good summary of all the re:Invent announcements, as well as the keynote videos.

Connecting to the SharePoint 2013 REST API from C#

Today I was updating an internal application we use for grabbing lots of Terminology data from SharePoint lists, and exporting it as TBX files for import into CAT tools etc.

This was required as the SharePoint on which it was hosted previously was upgraded from 2010 to 2013.

A small job I thought.

Then I discovered the the ASMX Web Service in SharePoint I used to grab the data previously, are deprecated in SharePoint 2013, probably not a surprise to anyone in the know, but SharePoint happens to be one of my pet hates, so development of it is not something that I tend to keep up to date with.

Anyway, I had to re-jig our application to use the SharePoint REST API, and I thought I’d provide the code here for connecting, as it look a little bit of figuring out.

The below (after you fill in your SharePoint URL, username, password, domain, and name of the list you want to extract data from), will connect and pull back the list contents to an XmlDocument object that you can parse.

XmlNamespaceManager xmlnspm = new XmlNamespaceManager(new NameTable());
Uri sharepointUrl = new Uri("SHAREPOINT URL);

xmlnspm.AddNamespace("atom", "http://www.w3.org/2005/Atom");
xmlnspm.AddNamespace("d", "http://schemas.microsoft.com/ado/2007/08/dataservices");
xmlnspm.AddNamespace("m", "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata");

NetworkCredential cred = new System.Net.NetworkCredential("USERNAME", "PASSWORD", "DOMAIN");

HttpWebRequest listRequest = (HttpWebRequest)HttpWebRequest.Create(sharepointUrl.ToString() + "_api/lists/getByTitle('" + "LIST NAME" + "')/items");
listRequest.Method = "GET";
listRequest.Accept = "application/atom+xml";
listRequest.ContentType = "application/atom+xml;type=entry";

listRequest.Credentials = cred;
HttpWebResponse listResponse = (HttpWebResponse)listRequest.GetResponse();
StreamReader listReader = new StreamReader(listResponse.GetResponseStream());
XmlDocument listXml = new XmlDocument();

listXml.LoadXml(listReader.ReadToEnd());

Localization of Email Campaign Content From Eloqua

Eloqua is a marketing automation platform, allowing marketers to easily create campaigns consisting of emails, landing pages, social media etc. via its ‘campaign canvas’.

Campaigns can be created via a WYSIWYG interface, allowing you to visualize marketing campaigns easily as you build them. It also integrates with CRM tools, automatically passing any lead data generated onto your sales staff.

My interest in Eloqua, and specifically its API, relates to the localization of email campaign content. This can be achieved manually by exporting the created email (as a standalone HTML file), then re-importing the localized versions post translation, creating a new version of the original, one for each language you have localized for.

Manual exporting of email content for localization is of course a perfectly valid approach, but the more languages you have, the more manual steps in this process, and the longer it takes, potentially tying up a resource.

The Eloqua REST API can be used to easily reduce the transactional costs related to localization of email campaign content. Using the API, you can quite easily automate the extraction of email content, and potentially send this content directly to your Translation Management System (TMS) such as GlobalSight or WorldServer, or straight to a translation vendor in the absence of a TMS.

The API documentation is pretty good. I also came across this samples project on GitHub released under the Apache license which I was able to use to knock up a proof of concept pretty quickly. It’s written in C# and includes functions to manipulate most asset types in Eloqua.

The email sample code in this library illustrates how to create, edit, and delete emails in Eloqua via the REST API. For example, it’s this easy to grab an Object that represents an email in Eloqua:

EmailClient client = new EmailClient(EloquaInstance, 
                         EloquaUsername, EloquaPassword, 
                         EloquaBaseUrl);

 try
 {
     Email email = client.GetEmail(emailID);
     return email;
 }
 catch (Exception ex)
 {
     // Handle Error...
 }

Some notes:

  • When retrieving the Email Object which represents an email in Eloqua, you need to specify the ID of the email in Eloqua. For automating the localization process, it could be difficult to determine this without user input. What I plan on doing is providing a nice UI so that users see only the emails that they have created in Eloqua (i.e. not all emails created ever), and can click on one and submit it for translation in one click.
  • The Email Object also contains other useful metadata like when the content was created, when it was last updated and by whom, the encoding, and the folder in which this email resides in Eloqua, useful for when you want to upload the localized versions.

So, that’s how easy it is to automate the retrieval of email content from Eloqua. The library I referenced also has support for other asset types like landing pages etc.

Next I plan on using ASP.NET Web API to turn this library into a HTTP service I can use to talk to Eloqua from other applications, such as the application that manages content submission/retrieval from our TMS.

GlobalSight Web Services API: Manipulating Workflows Programmatically

This is part four of my series of posts on the GlobalSight Web Services API. See below for the previous posts:

Here, I’d like to cover how you can manipulate workflows in GlobalSight via the API. For example, dispatching a workflow, accepting tasks in a workflow, or completing or moving workflows on to the next step.

This can be useful if you want to automate specific steps, or if you want to use an interface other than GlobalSight when dealing with workflows.

There are a couple of relevant functions that are pre-requisites when dealing with workflows programmatically:

  • getJobAndWorkflowInfo – This returns details about workflows in a particular job. The ID of the job in question is passed as a parameter to this function. This will return an XML response, detailing as well as the workflow information, some basic information about the job. We need this to get the ID of the workflow which we want to work with, within a particular job.
  • getAcceptedTasksInWorkflow – This will return the task information for a workflow, given the workflow ID (which we would have got from getJobAndWorkflowInfo). With the XML response here, we’ll be able to search for specific tasks in a workflow, and get the task ID – this is what is required in order to manipulate specific tasks in a workflow.

From the above, (I leave the parsing of the XML response as an exercise to the reader), we can begin to perform workflow tasks by using the task ID.

// Accept task
string acceptTask = client.acceptTask(auth, "63781");
Console.WriteLine("Accept Task Response: " + acceptTask);

// Send this worfklow to the 'Write to TM' step
string completeTask = client.completeTask(auth, "63781", "Write to TM");
Console.WriteLine("\n\nComplete Task Response: " + completeTask);

The above is a very simple example of accepting a particular task in a workflow (using the workflow ID), and sending it to the next step in the workflow via ‘Write to TM’.

Write to TM‘ here is the actual text on the arrow in the workflow diagram. I found this syntax strange, but it seems to work.

Capture

It should be noted that the user under which you are logged into the web service with must be on the list of those allowed to accept the task you are trying to accept, you will receive an error otherwise.

The GlobalSight Web Services API documentation has much more information the the types of actions you can perform on workflows, but the above should get you started.