Imagine you are driving but have not yet decided where you are going. Maybe you don’t have the slightest idea, or maybe you have a few options but don’t know which one you prefer. Why would you even start driving? But you started, so a few things can happen:

• You hesitate at every turn, and you invest continuous effort in decision-making. You can arrive maybe someone more or less decent, at a high cost, and perhaps not where you should be.

• You drive without minding the destination, maybe getting further away, maybe running in circles.

In any of these cases, there is motion but no progress, or there is no motion at all.

You can continue like this for a while, or you can, after some time, pull over and finally check the map, or give up and go home. You wasted time and effort because you lacked a clear vision and direction. I’m sorry to invalidate the fancy sentences on your wall, but that is not vision: vision must be something that tells you where to go and helps the high-level decision-making to get there.

Why you need clear vision and direction

Opposite to the above, when you go home, you may use slightly different paths every time, but you always arrive. You don’t need to memorize and repeat the steps. The specific turns are something you will figure out, and you can realign easily when needed -i.e., if you miss a step or one street is closed- because you don’t commit to steps, you commit to the destination.

That is a Vision: Do not obsess with the solution but with the problem to solve. Bricklayers think beyond the bricks; they think of walls, rooms, and entire buildings. Your team cannot work on single tasks but on a clear problem to solve.

There can be exceptions. As an entry level engineer it is fine if you think in terms of tasks and your work is transactional. But that’s just one stage in the journey, ideally one that should not last long. More on levels and career journey here:

If your team works one brick at a time, what you can expect to happen includes:

• They need you to be there to tell them what the next task is. There is not a natural flow-through or logical next step.

• They won’t be able to make any decisions because the context they have is limited to the task, and they don’t know how that contributes to something bigger. Which one moves more in that direction from any given two options?

• They won’t be able to prioritize a given list of similar tasks for similar reasons.

• They won’t know when something is done or not or when something relevant is missing. They can evaluate the acceptance criteria as written down, but they will be unable to zoom out and see if It really contributes to the bigger thing we want to achieve.

• They will not be able to think ahead, propose new ideas, features, or experiences, or contribute to the specific thing we want to solve or improve for customers other than by mechanical execution.

All that results in micromanagement and handholding, underutilization of talent, wasted efforts, misaligned results, and inefficiency. And all this can be solved by setting a clear vision, giving direction, and aligning towards common goals.

Setting a clear direction and aligning cross-functional teams towards common goals.

You may want to keep some of these steps in mind.

1. Define a Clear, Shared Vision:

   • Define a unified vision statement/narrative aligning with the organization's objectives. You cannot make it up. It must be something you are trying to achieve.

   • Communicate and explain it, and ensure that all teams understand how their work contributes to this larger goal. It is not on the team to understand it but on you to connect the dots. Remember that it needs to be actionable, so it needs to be real.

2. Set Common Goals:

   • Establish specific, measurable goals that apply across all involved teams and clearly contribute to the vision - or at least align with it.

   • Communicate these goals and make sure everyone understands them. Prepare yourself to answer questions, reflect, and iterate when you don’t have answers.

3. Involve Key Stakeholders Early:

   • Engage leaders and representatives from each functional team in the goal-setting process and strategy definition.

   • Gather input and address concerns to ensure compatibility with reality and buy-in from all teams.

4. Promote Open Communication:

   • Encourage transparency and sharing of information to avoid silos.

   • Is there any new project? Is there a change in direction? Are we pursuing something different? Are priorities changing?

   • It doesn’t mean that everyone has to know everything leadership is discussing, but when something is decided, people must know, and when something will affect everyone significantly, key people must be consulted and their input incorporated.

5. Clarify Roles and Responsibilities:

   • Ultimately, everyone has to understand that everyone is part of the same mission but ensure that everyone understands their individual and collective responsibilities.

   • Clarify both boundaries and responsibilities that may not be so obvious. Are software developers required to do reports or take care of deployments, and if so, why? What is your objective, and what is expected of you?

6. Foster Collaboration:

   • Let teams understand how their different pieces of work contribute to something bigger. For example, software developers can enjoy working closely with customer service and operations and understanding how their products exist in real life.

   • Create collaboration opportunities for teams, such as joint projects or task forces.

   • Build trust by encouraging teams to share resources, knowledge, and expertise.

7. Provide Leadership and Support:

   • Ensure that leadership is actively involved in guiding the teams and resolving conflicts.

   • Do not give just a solution, but explain the rationale. Why was this chosen over this? Why that one was not an option? What are the guiding principles leading decisions?

8. Monitor Progress and Adjust:

   • Regularly review progress towards the common goals, use as much relevant data as possible, and do not use irrelevant data. For example, don’t measure developers by the number of commits: it can be faked and doesn’t contribute to any goal. Measure how many times they solved the problem you still didn’t tell them how to solve, or they were one step ahead in the right direction when you didn't even know the next step.

   • Address any misalignments or challenges promptly to keep teams on track. Understand where the misalignment comes from: ensure the direction is understood, relevant, and still makes sense.

9. Align Incentives and Rewards:

   • Design incentives that encourage collaboration and align with the achievement of common goals. Your success is not your task but how that task contributed to something bigger.

   • Recognize and reward teams and individuals who contribute to cross-functional success.

10. Celebrate Successes Together:

    • Reinforce the value of working together towards shared objectives by being explicit about what those objectives are.

    • Acknowledge and celebrate milestones and successes as a unified team, being explicit about what you are celebrating.

Final notes: Keep it real

It is very engaging to walk towards a vision and contribute to something that has an impact on the world. But detecting misalignment between narratives and reality causes the opposite effect, so there are key aspects to consider while defining your vision:

• Make sure the vision is genuine and not just epic-sounding words. It has to be genuinely aligned with something the company is making continuous and proactive efforts to achieve.

• Ensure your team understands, buys, and identifies what it means and what aspects of their work contribute to it. Be interactive, listen, give room to be challenged, and be equipped with genuine answers.

• If the org is big, the teams' work can seem too far disconnected from that vision. Create a vision for the team that they identify with because it is easier to map to their work, and that clearly contributes to the overarching vision.

• Give direction that aligns with that vision, is consistent, and is compatible with everyone’s day-to-day work, and that proves to be real when the time to celebrate arrives.

There is a rule of having a rule of not deploying on Friday. When you join a new company, you wait to hear about the rule of no Friday deployment. If they have the rule, then they are compliant with the universal rule of having that rule.

But what happens with the ones that don’t have that rule? Are they reckless? Are they unaware of the rule of having that rule? Or do they know something that the others don’t? We will explore that idea.

Before we start, there is an anecdote that comes to my mind when thinking about deploying on Friday. For a few years, at the beginning of my career, I was a freelancer in Barcelona. I made websites and apps and whatnot. I was my DevOps and my on-call. Soon I started cooperating with a company in which, for the first time, I heard that rule of no deployments on Friday. I was genuinely confused. Is not Friday the best day? I honestly preferred Friday because if something fails, I could quickly fix it on Saturday, while during the week, I had a million of competing priorities. That was me being pragmatic and knowing zero about the corporate world.

Then I learned how wise advise it was, and how good it was to spend the weekend in the beach.

Friday's releases were, historically, hands down, a weekend wrecking ball.

But why, why?

I remember the time when a deployment could seriously break things. I saw teams spending a couple of days literally trying to fix the mess that was introduced in a minute and could not be rolled back: Commits piled up for months, and that broadly unknown entanglement was thrown on top of the working production code. Different systems were deployed in a specific order to ensure dependencies were met. Configuration manually changed. Dependencies leak-ins and other monsters causing the “It worked on my machine ¯\(ツ)/¯”.

CI/CD was more hype than reality. Canaries, blue/green deployments, one-click rollbacks, and feature toggles were not a thing.

But that’s not the case anymore, is it?

What does a deployment look like today?

Note: if yours doesn’t look like that, you have some food for thought right here.

• Changes are deployed frequently. You don’t ship months' worth of work at once; you just need smaller changes.

• The build can take some time (which is fine because you are definitely separating the build and run stages), but the release happens almost immediately.

• Infrastructure, dependencies, and the jungle of libraries, versions, and server software are managed as code in an imperative/declarative way, drastically reducing the number of things you need to know and remember to do to replicate things from one environment to another.

• You may have feature toggles, and behavior changes will not actually happen at deployment time but when you decide to enable them. And nothing will continue to happen if you don’t want it anymore.

This not only makes it easier to fix things or revert if something goes wrong but also makes it harder to break things in the first place. Also, if you have the right observability, dashboards, metrics, and alerts and document beforehand the post-implementation validations…, you also know right away if something went wrong. This means we will have a relatively low “Risk Priority Number”.

Risk Priority Number (RPN)

In Failure Modes and Effects Analysis (FMEA), the Risk Priority Number (RPN) is a critical metric used to prioritize risks based on three factors:

• Severity (S) is the impact of a failure. This is the consequences or seriousness if it occurs without considering any mitigating actions or controls.

• Occurrence (O) estimates the likelihood or frequency of it occurring.

• Detection (D): The ability to detect it before it reaches the end user. It considers the effectiveness of current controls or mechanisms to identify the issue.

For example, if an issue could render the checkout process unusable, the Severity would be high. But if the chances of occurrence are very low and the chances of detecting it right away are next to 100%, the overall risk is next to nothing, more so if reverting it would be one click.

So, should I go ahead and deploy on Friday? Wait, hold your horses. Let’s see the pros and cons.

Against Friday Deployments

What popular beliefs are:

1. Risk of Downtime: If issues arise, they may not be addressed promptly, leading to prolonged downtime, especially if staff are unavailable or reluctant to work over the weekend.

2. Work-Life Balance: Teams may be forced to work late on Fridays or over the weekend to resolve deployment issues, which can negatively impact morale and work-life balance if it happens often -often being a subjectively defined word-

3. Limited Resources: Even with on-call teams, available staff and support resources might be reduced over the weekend, making it harder to manage complex issues. What if you depend on marketing, content, legal, or other teams that typically don’t have an on-call rotation?

4. Customer Impact: The response time to any incident may be longer, which may disrupt customers who rely on those services during the weekend, leading to dissatisfaction and potential revenue loss.

5. Time to detection. What if there is a silent problem? The time it takes to discover and diagnose a problem can be longer during the weekend due to not having everyone doing their everyday work, testing flows, checking dashboards, etc.

6. Operations and Customer Support. Even if everything goes well, if you are launching a new product and, for example, it requires special training, maybe you prefer to avoid a weekend when your CSR may not have team leaders available in case of any confusion or customer queries.

Counter arguments

• If you are afraid of Fridays, you should be fearful of Thursdays and every day after 3 pm. If you break something on Thursday, you will find out maybe on Friday afternoon, and then what? Would you be deploying on Friday evening to fix the Thursday bug? Okay, fine. No Friday, no Thursday, no afternoons after 3 pm. Wouldn’t it be better if you arrived at a safe way of deploying any day?

• Downtime why? What the heck are you deploying? If you deploy daily as soon as you have some tested updates to deploy, all your deployments are incremental and small, and if you stick with smaller changes, there isn't much that can break.

• Why customer impact? Does it mean you found an error and didn’t mitigate it immediately? Or was it not straightforward to diagnose or mitigate it, so you did not roll it back? Or you didn’t detect it? Does this mean that you deployed changes without alerts and dashboards? Well, your problem is not the deployment window but your definition of done.

Some net benefits of Friday deployments

1. End-of-Week Momentum: Teams often push to complete work before the weekend, potentially increasing productivity and the completion of key tasks.

2. It avoids confusion and frustration. A team has been working hard on a project and is ready to deliver, but they may miss a deadline because it is Friday, and they are not allowed to release it, even when they see no risk.

3. Resource Availability: If a company has a robust on-call system, deploying on a Friday can ensure that resources are available to address issues during the weekend rather than disrupting the workweek.

4. Adapt to traffic patterns. Depending on the usage patterns of your customers or from what time zones they visit, Friday or even the whole weekend may actually be a quieter period and, thus, more suitable for updates.

5. Foster a culture of readiness and adaptability. Muscles have to be trained. If you don’t usually deploy on Fridays, you will be in trouble if one day you need to deploy something on Friday, let’s say a bug fix or a compulsory update due for example to regulation or market reasons. That would be riskier and more stressful for everyone than if it were a standard practice:

   • Are you going to put in place a special process?

   • Do you have a sophisticated enough branching model, or are you able to toggle off all the changes that are not meant to be promoted?

   • If you don’t usually have an on-call or it is not enough to cover this, do you need to coordinate with HR and Legal and agree on a compensation model, select and influence some team members to be on-call, create runbooks, and draft an operational plan?

   • And the hardest part: the cultural aspect. If the common understanding is that Friday deployments are evil, there is a negative halo and pressure around it. A team that has to deploy to meet a mandatory deadline or fix a bug, aside from all the problems and pressure, will also need to prepare an excellent narrative to justify it and react to complaints and criticism.

6. The vicious circle of mitigating risks by freezing actions. Risk aversion is understandable, and a code freeze can be an easy escape, but as with all easy escapes, it can be abused. No deployments to production in specific periods, then UAT or stage is blocked because it is ready with something that has to be released to production after the freeze. QA is blocked because something else has to go urgently to UAT after UAT is available again. Code piles up and new arbitrary freezes are introduced to secure complex releases. A never-ending chain that will end up in extended periods with no deployments, hotfixes in certain environments without coming from the lower one, environment swapping, etc. And the inability to perform urgent deployments without breaking all the rules and invalidating all the setups. The solution becomes the problem.

7. The smaller the increments, the lower the risk. Piling up commits, whatever the reason, is always wrong. Deploy as soon as possible and create your processes to take yourself in that direction.

With all this, it sounds better to deploy as early as possible and treat all days equally.

But what if I there is bigger, risky change? 

On certain occasions, you have one-way-door, drop-the-bomb kind of changes. Something hard to revert or toggle off if something goes wrong.

For those, you probably need a different preparation and rules than for a more mechanical business-as-usual deployment.

In this case, you may need a specific preplanned and broadly aligned window. But then, in this case, the day of the week is probably the least of the concerns. Maybe you will even have someone out of working hours doing that, like a Saturday evening, or the opposite can be true: if you need a specific set of people or it is highly unsafe or unpredictable, you may want to wait until Monday. So, in these cases it will be more of a case-by-case situation than a rule.

Wrapping all this up

No Friday deployments is like saying :my knee hurts when I touch it, so my solution is not touching it”. Sometimes, things are a symptom of a more fundamental issue, and that issue is what you should be looking at.

Dive deep to understand why the deployment process feels scary and address the root cause. Build confidence. And then deploy whenever you need to.

Addressing the root cause doesn’t usually need reinventing the wheel, but a few known things to consider.

The foundation for a safe deployment

You may have in mind some long-term fancy things, but the reality is that the most relevant ones are essential solutions that are relatively easy to put in place:

• Deploy frequently to keep changes small. The bigger the difference, the bigger the risk.

• Use feature toggles for all your developments. This will help in two ways: to hide partial developments that are not supposed to reach users yet, and to deactivate a feature when the deployment goes wrong. In the future, it can also help with more things, such as progressive rollouts, A/B testing, etc.

• Design and document a quick and easy rollback process for failed deployments.

• Add to your process smoke/regression testing before the release.

• Instrument Real User Monitoring (RUM) and application monitoring (APM).

• Create dashboards and alerts to monitor relevant functional metrics (i.e., are there anomalies in the number of onboardings? are errors trending up?)

When you have theses, or a good number of them, you should be able to deploy any day of the week. If you cannot have that, you can rather avoid deploying on Friday, but that definitely means that you need to evolve your processes, and that should be your top priority because it is not fancy stuff that you are missing, but basics.

Plan your deployment for sucess

All this applies to Fridays, but also to any day for that matter.

• Deploy early, ideally before noon or at least a few hours before you finish the day. You want to have time for testing and rolling back, and you don’t want to leave late if it takes too long.

• Avoid last-minute changes or rushing through the deployment process due to the eagerness to close things before you go. As you would do any other day, deploy only if everything is ready. Have the testing sign-off, go/no-go, or readiness review and everything in place early in advance.

• Prepare a clear deployment plan: list the steps, align the people that are required, prepare a backup and rollback plan, etc.

• Communicate clearly: inform any relevant part about the deployment schedule, then plan and any potential risks involved.

The extra mile

Nothing else is really needed to enable Friday deployments and a healthy cadence. With that said, you may want to add some additional sophistication in the longer run. Even major changes like framework upgrades, vendor migrations, etc., could be done safely and with little effort with the right things in place. You may need to invest a bit more in automated testing and multi-phased rollouts, but it takes a lot of drama away once you get there.

• Automatic regression testing for all features.

• Full coverage end-to-end production testing.

• A full-fledged CI/CD pipeline, where every PR to prod is automatically deployed after a successful run of tests.

• Safest release models:

  • Canaries: rolling out progressively to a number of customers, monitor (errors, functional metrics, latency, etc), and if everything goes well for a period of time, increase progresively the percentage

  • Blue/Green deployments: Keep the old and the new versions in parallel and just let the load balancer route traffic to the new instances. If things go wrong, you route all traffic to the old instances again.

Summary

Not deploying on Fridays made total sense in a time when deployments were rarely done and risky. In today’s world, with more frequent deployments and better ways of detecting issues, toggling features off, or rolling back deployments, having deploy freezes introduces artificial limitations and can paradoxically increase the risk of introducing defects or being less ready for detection or mitigation.

Removing the deployment freezes altogether can reduce the complexity of your deployment processes and also improve your responsiveness to incidents by looking at the challenges and risks and preparing for them rather than pretending that risk can be dodged forever.

Ultimately, the decision to deploy on Fridays depends on various factors, including the deployment pipeline's maturity, the product's nature, the availability of staff, and the company's risk tolerance. Many companies avoid Friday deployments to mitigate potential risks, while others might have the infrastructure and processes to handle them effectively.

Having or not thorough testing, monitoring, and rollback strategies can be the main drivers for this decision. If you do have that, you can do deployments any day. If you don’t, this is giving you a very actionable hint of what your actual problems are, and you must evolve your processes until your answer is positive.

In the first part, we explored fundamental practices for identifying and mitigating biases.

In this part, we will see a comprehensive list of common biases explained in detail and specific strategies for interrupting them

Common biases in hiring and specific identification and mitigation strategies

1. Stereotyping, Education, Experience, and Cultural Bias

Description

Making broad assumptions about groups of people without considering individual differences leads to oversimplified views. This includes judging a person or situation based on previous experiences with someone with similar characteristics and believing that our beliefs, culture, or processes are the "correct" or "normal" ones.

Examples

• "He is from university X. You know what standards they have…”

• "I don’t want to hire people from X kind of company."

• "He worked at Google. Definitely, he is a good candidate."

Mitigation Strategies

• Use multiple sources to build objective assessment criteria and establish alignment on the requirements.

• Collect both supporting and dissenting data when designing assessment criteria.

• Challenge assumptions to draw out facts, evidence, or examples.

• Ask follow-up questions when you hear generalizations or stereotypes.

• Question your doubts about someone's abilities by evaluating actual anecdotes.

• Challenge assumptions and seek facts and evidence.

• Reflect on how your personal experiences and cultural norms shape your perceptions, and ask follow-up questions when encountering generalizations or stereotypes to try to invalidate your own perspective.

2. Halo/Horn Effect

Description

Attributing a general impression based on a single characteristic or a first impression. The halo effect is the tendency to form a positive impression of someone based on one favorable trait, while the horn effect is the tendency to create a negative impression based on one unfavorable trait.

Examples

• Halo Effect: An exceptionally well-dressed and articulate candidate is perceived as highly competent in all aspects of the job, even without thoroughly evaluating their skills and experience.

• Horn Effect: The interviewer perceives a candidate who arrives late as lazy and unmotivated.

Mitigation Strategies

When you see yourself jumping to conclusions very quickly or fast-forwarding some interview questions, challenge yourself:

• Can I get evidence that the specific behavior is consistent and not a one-off?

• Have I created a general opinion based on a particular thing?

• Can I ask questions to try to prove any of my assumptions wrong?

3. Descriptive and Prescriptive Bias

Description

Descriptive bias involves interpreting information based on how things are supposed to be, often influenced by stereotypes, while prescriptive bias involves enforcing norms and expectations about how things should be or how individuals should behave. When someone goes against those norms, they may be viewed negatively.

Examples

• The interviewer believes that women should be nurturing and collaborative, and therefore rates a woman who didn’t show strength in teamwork and empathy negatively while not holding male candidates to the same standard. This prescriptive bias enforces assumed gender roles and expectations, potentially disadvantaging candidates who may possess other valuable qualities relevant to the job.

Mitigation Strategies

• Am I starting my interviews with a blank slate, abstracting myself from what I can assume from the candidate?

• Do I have calibration standards that apply to any candidate for the role and level?

4. Similarity, Affinity, and In-Group/Out-Group Bias

Description

It is common to prefer people with similar interests, skills, or cultural backgrounds who are like us or remind us of ourselves. We put those in a favorable “in-group” and everyone else in a less or not-favorable “out-group.”

Examples

• "I don’t think that person is a good fit."

• "I don't think she'd understand us."

• "They had a poor communication style" to a candidate from a different country based on their accent.

• The candidate uses the same programming languages I love.

• The candidate lived in the city I used to live in.

Mitigation Strategies

• Would I provide the same feedback if the person belonged to a different demographic group or had different experiences?

• Am I overly focusing on specific aspects of the candidate’s background or personality?

Challenge yourself to get more data:

• What answers did they provide that helped you draw that conclusion? For example, If communication was poor, why would you say so? Can you point to specific reasons? Are you influenced by the fact that the individual has a different native language?

• Is the candidate just saying or doing things in different ways that may seem unfamiliar to you but not necessarily worse?

• Is the fit about a clash with your existing team members or the company as a whole? Can those specific particularities be learned? For those of us who have been in the company for a while, remember that you also had to adapt when you joined.

5. False Consensus Effect

Description

Overestimating the extent to which their beliefs, values, opinions, or behaviors are typical and shared by others. This bias leads people to assume that others think and act the same way they do, often resulting in a distorted perception of social norms and consensus.

Examples

• An interviewer strongly values a highly competitive work environment and assumes that all top candidates also prefer and thrive in such an environment. During the interview, the interviewer might frame questions and interpret responses through this lens, assuming candidates who prefer collaboration or a more balanced work-life dynamic are less committed or ambitious. This bias can lead the interviewer to favor candidates who share their competitive values, potentially overlooking highly qualified candidates who might excel in a different but equally effective work environment.

Mitigation Strategies

• Acknowledge the potential for this bias and reflect on personal beliefs and assumptions.

• Prepare a calibration guide before the interviews and ensure it is validated and broadly used to get a more certain idea of what the actual consensus is.

6. Distance Bias

Description

Distance bias reflects our instinct to prioritize and trust what is closer to us, whether in terms of physical proximity, time zone, or temporal immediacy. This bias can influence relationships, decision-making attention, and perceptions of performance, as individuals working in the same location or office often develop stronger bonds and may be viewed more favorably compared to those who are further away.

Examples

• A manager favors a team member they see daily in the office over a remote colleague.

• A candidate is preferred because he lives in the same city or the interviewer has common friends.

• During evaluations, employees in the same time zone receive more positive feedback than those working remotely from different time zones.

• A leader gives more critical projects or tasks to team members located in the same building, believing they're more reliable due to their physical proximity.

Mitigation Strategies

Reflect on whether proximity influences your perceptions and strive to value distant colleagues equally.

• Do I have a closer personal connection with this person than any other?

• If you already work together, does the frequency with which I see a person influence my perception of others more distant?

• Would I value others more if they were closer to me?

7. Recency Effect and Availability Bias

Description

Placing greater emphasis on or giving more attention to recent events and observations over those from the near or distant past (recency). Depending on the information that is most easily and quickly recalled (availability). Making decisions based on the most recent or readily accessible data and anecdotes rather than gathering comprehensive information.

Examples

• "The last/first candidate was the best one."

• "I don't want to work with this person on this; last time, he made a big mistake."

Mitigation Strategies

• Take notes from candidates and compare everyone against your calibration guide—make sure you have one— and not against each other.

• Don't make up your mind on the spot.

• Ask follow-up questions to probe and challenge the candidate and yourself.

• Can you find examples from the past that invalidate your last perception positively or negatively, or am I overemphasizing last achievements or failures?

• Are you looking at a comprehensive picture? Do you have evidence that good or bad behaviors are consistently repeated over time?

• Did you give yourself enough time to consider all relevant information before making a quick decision?

• If you are taking several interviews on the same day, get breaks to release your cognitive fatigue or information overload.

8. Sunk Cost Fallacy

Description

Refers to the tendency to continue investing in something or someone (a project, task, employee, strategy, decision) based on the cumulative prior investment (time, money, resources), despite new evidence suggesting that the cost, moving forward, outweighs the expected benefits. It's driven by a desire not to waste the sunk costs, even if continuing doesn't serve the best interests of the project or the individual.

Examples

• "I know this is taking longer than expected, but we are almost there."

• "Even if it is not a priority anymore, we have made significant progress; we should just go ahead to avoid frustrating the team."

• I'm not sure about the candidate, but given the two months in the process and the time of so many interviewers and recruiters, I think we should just close it and send an offer.

Mitigation Strategies

• Are we making an informed/calibrated decision or trying to avoid conflict or the perception of loss?

• Do we have quantitative reasons for continuing? Is this still aligned with our objectives?

• Are there conflicting opinions in the group that can open our eyes to unseen aspects?

• For projects, prepare your Business Requirement Document documents to incorporate reasons to pursue and not pursue an idea beforehand, with relevant context and data, acceptance/exit criteria, and a clear stop loss.

• For employees/candidates, create role leveling and calibration guides so you have clear criteria for hiring and not hiring beforehand. Do the same for the probationary period and performance reviews, and drive your unregretted attrition processes.

9. Affect Heuristic

Description

The affect heuristic is a mental shortcut in which people make decisions and solve problems based on their emotions and feelings rather than on objective analysis and facts. This heuristic plays a significant role in how we assess risks and benefits in various situations, often leading to quicker, but sometimes less rational, decision-making. Specific feelings and perceptions come from the gut and are prone to unconscious bias. The candidate may remind the interviewer of someone else or a past situation, or it may be something built about specific traits.

Examples

• "I just felt a good vibe from her right away, which makes me think she’d fit perfectly into our team."

• "He seems really confident and likable, so I believe he'd handle the job's challenges well."

• "Her presence was a bit imposing, and he made me feel uneasy."

Mitigation Strategies

• Ask for specifics about what actions or statements or actions prompted these observations. What specifically did they say or do to make you say that?

• Did other interviewers feel the same way?

• Try to probe yourself to identify if there is a reasonable root for your feelings that are not related to the candidate.

10. Conformity Bias, Groupthink, and Social Cohesion

Description

Conformity bias occurs when individuals adjust their opinions and behaviors to align with what they perceive as the group consensus. In workplace settings, this can lead to a culture where new or differing ideas are less likely to be expressed or valued, particularly if the emphasis on fitting into the existing culture is strong. Groupthink and social cohesion are team dynamics closely related to conformity bias. These dynamics can significantly impact decision-making processes within teams, often in ways that compromise the effectiveness and creativity of group outcomes, preventing innovative ideas and progress. Groupthink prioritizes group harmony over critical evaluation, leading to poor decisions when dissent is suppressed to maintain cohesion. Social cohesion strengthens team bonds and can enhance collaboration and satisfaction, but it may also prevent the expression of diverse or critical viewpoints, leading to unchallenged and potentially suboptimal decisions. Both dynamics highlight the need for managing teams in ways that balance unity with the encouragement of diverse opinions and critical discussions to mitigate the effects of conformity bias.

Examples

• The hiring manager is inclined to hire, so everyone is inclined

• Most people didn’t like the candidate, so I won’t be the one trying to defend him even if I saw great potential

• The interviewers like a specific approach from the candidate; based on my background, I know it is wrong, but they won't understand, so I don’t mention it.

Mitigation Strategies

• Create psychological safety to allow open discussions among the team and remove the potential risk of conflict or feeling like an outlier by avoiding penalizing when it happens.

• Explicitly accommodate those challenging group-thinking and opening up perspectives.

• Proactively seek input from all team members.

• Be intentional about starting with lower-ranking or less senior people so they can express their opinions without getting biased by the leader, feeling that's what needs to be followed, or feeling discouraged to speak up if they have opposing opinions.

Conclusion

Cognitive biases are systematic deviations from rationality, influenced by personal preferences, team dynamics, and urgency to fill a role. These biases can significantly impact the hiring and interviewing process, skew assessment, lead to unfair and irrational judgments, and result in flawed hiring decisions.

Understanding and mitigating these biases is crucial to ensuring a fair and effective interview process. This will allow candidates to be evaluated based on merit rather than unconscious prejudice. For that, you should make a proactive effort to understand the specific common biases and strategies to interrupt them.

This series explores:

• Part 1: General strategies to prepare yourself and your processes for general bias identification and mitigation

• Part 2 (coming soon): A comprehensive list of common biases explained in detail and specific strategies for interrupting them

Fundamental practices to prepare the ground for avoiding biases

Disrupting the effects of bias in a job interview requires a structured, objective, conscious, and proactive approach. Implementing these strategies can help improve decision-making and ensure a more objective and fair assessment of candidates and hiring decisions.

Preparing a self-aware and Inclusive Culture

Self-awareness and reflection

• Reflect on your own biases and understand how they influence your perceptions and decisions.

• Consider how your cultural background and personal experiences shape your views.

• Acknowledge the potential for this bias and reflect on personal beliefs and assumptions before the interview process begins.

• Encourage self-reflection and awareness of personal biases before and after interviews.

Training and Awareness

• Participate in training programs, take courses, or watch videos to learn about and recognize various cognitive biases. YouTube can be a good resource.

• Participate in diversity, equity, and inclusion training programs.

• Learn strategies to mitigate these biases in decision-making processes. More on this will come in the next post in this series.

• Read on biases (the links are to read for free in Audible with a free trial)

  • ”Thinking, Fast and Slow" by Daniel Kahneman. This book comprehensively explores the two systems of thought: fast, intuitive thinking and slow, deliberate thinking. Kahneman, a Nobel Prize-winning psychologist, explains how cognitive biases arise and affect decision-making, including in hiring.

  • "The Undoing Project: A Friendship That Changed Our Minds" by Michael Lewis. This book chronicles the collaboration between Daniel Kahneman and Amos Tversky, whose work on cognitive biases revolutionized the understanding of human decision-making. It provides valuable insights into how biases can impact hiring decisions.

  • "Predictably Irrational: The Hidden Forces That Shape Our Decisions" by Dan Ariely. Ariely explores the irrational behaviors and biases that affect our decisions. His insights can help hiring managers recognize and mitigate irrational biases in the hiring process.

  • "Blindspot: Hidden Biases of Good People" by Mahzarin R. Banaji and Anthony G. Greenwald delves into implicit biases and how they unconsciously influence our perceptions and decisions. It's particularly relevant for understanding and addressing hidden biases in hiring.

  • "The Power of Noticing: What the Best Leaders See" by Max H. Bazerman focuses on how leaders can improve their decision-making by being more aware of cognitive biases and offers practical strategies for noticing and mitigating biases in various contexts, including hiring.

  • "The Art of Thinking Clearly" by Rolf Dobelli is a collection of short chapters on cognitive biases and logical fallacies. It is a practical guide for recognizing and avoiding biases in everyday decision-making, including hiring.

• Educate yourself on different cultures, traditions, and perspectives. A good book on this is The Culture Map by Erin Meyer

Inclusive Practices

Promote and practice inclusivity in your personal and professional life, and create environments where diverse voices are heard and respected.

• Diverse Hiring Practices: Implement inclusive hiring methods. Think about balance in your sourcing, every step of the funnel, and the promotion process.

• Mentorship and Support: Mentor individuals from underrepresented groups.

• Equal Opportunities: Ensure equal career development opportunities.

• Inclusive Communication: Encourage open and respectful communication.

• Flexible Work Policies: Implement flexible arrangements to accommodate different needs.

• Accessibility: Make the workplace accessible to all.

• Celebrate Diversity: Recognize and celebrate diverse cultural and religious events.

• Feedback Mechanisms: Establish channels for feedback on inclusion efforts.

• Measure Progress: Set and measure diversity goals regularly.

• Stay Committed: Demonstrate a long-term commitment to inclusivity, continuously learning and adapting strategies.

Prepare your interview processes

Structured Interviews

• Develop a clear and detailed job description outlining the required skills and qualifications.

• Develop a set of standardized questions that focus on job-relevant skills and competencies rather than personal preferences or values.

• Ensure these questions are designed to assess the candidate's qualifications objectively.

• Use consistent questions for all candidates to ensure a fair comparison.

Objective Criteria

• Creating structured approaches to decision-making, such as checklists or decision matrices, can help ensure that emotional responses are replaced with factual analysis.

• Utilize leveling guidelines for hiring and promotion audit guidelines during promotion decisions. 

• Create clear, objective assessment criteria for evaluating candidates based on the job's specific requirements.

• Define calibration guidelines per competence to assess candidate responses with a clear, predefined idea of good or bad behaviors.

• Ensure that these criteria are inclusive and consider multiple perspectives.

• Rate each candidate against these criteria consistently rather than relying on overall or subjective impressions.

• Where possible, anonymize parts of the application process to focus on skills and experience rather than personal attributes or assumed preferences. For example, it is controversial, but the phone screener or interviewer doesn't need to be familiar with the candidate’s CV before the interview, as it can create bias. Your questions will be for the role, not for the candidate. 

To help structure your objective criteria, if you are in the software development industry, you may find useful my last article:

Seek diverse perspectives

• Involve multiple interviewers from different backgrounds in the hiring process, including industries, teams, levels of experience, gender, etc, to gain diverse perspectives and provide a balanced evaluation of each candidate.

• Engage with people from different backgrounds, cultures, and experiences to broaden your perspective while defining your process.

• Seek out and value diverse viewpoints in discussions and decision-making processes.

During the interview

Note-taking and documentation

• Take detailed notes during the interview, focusing on specific answers and behaviors related to job performance and using the language stated in your calibration guides.

• Curate notes objectively, separating facts from personal impressions.

Follow-up questions to challenge assumptions

• Ask follow-up questions to probe deeper into responses and clarify any assumptions. You may want to prepare a list of likely follow up question to probe and challenge. In most cases the answer will hint you the follow up question, but also you want to make sure you don’t forget important, necesarry ones.

• Ensure you understand the context and specifics behind each answer. Again do this by asking more questions to narrow down the context and eliminate blind spots. Do not assume anything, even when you may think your assumtion can be the only possible one, it may be wrong.

• Challenge yourself as well as the interviewer. Try to disconfirm your first impressions. You can make up your mind in the first minutes of the interview. Ask questions to try to prove your first impression wrong.

• Actively question and challenge stereotypes and biased statements when you encounter them.

• Encourage others to provide facts, evidence, or examples to support their claims.

After the interview

Panel debriefing and data-driven decisions

• Debrief with all the interviewers.

• Discuss each candidate's responses collectively and compare evaluations to mitigate individual biases.

• Rely on data and evidence rather than assumptions or stereotypes.

• Analyze both supporting and dissenting data to form a balanced view.

• Engage with people from different backgrounds, cultures, and experiences to broaden your perspective while defining your process.

• Seek out and value diverse viewpoints in discussions and decision-making processes.

• Feedback should be objective, job-related, and focused on the candidate’s demonstration of the job-related knowledge, skills, and experiences required for the position.

• If someone isn’t able to share specifics about what the individual said or did to form the interviewer's opinion, then be careful whether to include that feedback in the decision process.

• Start seeking input from lower-ranking or less senior panel members so they can express their opinions without getting biased by the leader or feeling discouraged from speaking up if they have opposing opinions.  

Feedback and Continuous Calibration and Improvement

• Regularly review and calibrate your interview process to ensure it remains fair and unbiased.

• Seek feedback from other interviewers and candidates about the interview process and decisions to identify any biases that may have influenced outcomes and potential areas for improvement.

• Continuously review and adjust interview practices to ensure fairness and objectivity.

• Regularly review and update your practices to ensure they remain unbiased and inclusive.

• Seek feedback from colleagues and be open to making changes based on new insights and understanding.

Conclusion

Removing biases from any of your processes, such as hiring, is not a reactive action but a series of proactive and consistent habits and approaches to foster a culture, awareness, mindsets, and preparation that will enable you to make the best and more objective decisions.

Implementing some of these strategies will help ensure a more objective and fair assessment of candidates and hiring decisions, fostering a more inclusive and equitable environment.

Talent acquisition had asked us for help defining the differences among levels for a software development engineer. That is like an invitation to a birthday party with free pizza and drinks: pure enjoyment with no downsides. But then my manager said: easy, for juniors we want three years experience, for seniors, five. My soul wept a little.

What could I do but challenge that? In the single minute that I had the floor -as a new joiner at that time- I did my best to explain my point of view: That is not okay. There are differences in responsibilities and expectations, and we should discuss different dimensions if we want to have significant level definitions that contribute to a functional organization in the short and long term.

Years of experience are nothing alone. I recently wrote an article about this:

Even further, not even a current title means anything by itself. Validating internal or external hires based solely on their previous titles or CVs is ineffective for two reasons:

• The same title can be very different across different companies,

• Holding a position does not necessarily equate to being capable of fulfilling a relevant portion of its various responsibilities.

I have also written an article on this topic:

Let’s talk about what matters per level, but before that, let’s see how many levels we can talk about.

How many levels for a role should we consider?

Talking about juniors and seniors alone, as many people do, is a gross oversimplification. There are infinite opinions about this, but certainly, there are not only two levels. We can easily talk about 4 or 5 different seniority levels that are easily applicable to any company, and that is thinking about the individual contributor path; most of those levels could branch out to an equivalent in the management side.

I will discuss levels, and I'd like to challenge myself by creating a comprehensive scale that encompasses a broad spectrum. You can combine items if you need fewer or interpolate/extrapolate if you need more. The levels I will be using are:

1. Entry-Level, Junior.

2. Mid-Level, Semi-Senior.

3. Senior Engineer

4. Principal Engineer

5. Senior Principal

6. Staff Engineer

7. Distinguished/Fellow Engineer

In some companies, there can be additional individual contributor roles, such as architects or enterprise architects. In contrast, in others, the architect role will be equivalent to Senior or Principal engineer, being these the teach leads taking care of solution or technical designs or review the designs of others. I don’t want to get lost in the accuracy of the naming of levels, but I primarily draw a scale and focus on describing the differences at all levels in that scale.

What are the dimensions to look at?

Here’s a list and a brief explanation of all the dimensions I consider when evaluating a person against a role and level.

1. Problem-Solving and Complexity: The ability to identify, understand, and resolve issues, including the level of complexity of the tasks or problems handled.

2. Autonomy and Execution: The degree of independence in carrying out tasks and the effectiveness in completing them.

3. Technical Skills: The depth and breadth of knowledge in programming languages, tools, frameworks, and technologies.

4. Leadership and Mentorship: The ability to lead projects and teams, as well as mentor other engineers.

5. Impact: The extent to which the individual’s work affects the team, project, organization, or industry.

6. Communication: The ability to convey ideas effectively to peers, supervisors, and stakeholders.

7. Ambiguity: The capability to handle uncertain situations and find clarity.

8. Process Improvement: The ability to identify inefficiencies and implement improvements in processes.

These dimensions provide a comprehensive view of the expectations and responsibilities at each level within the software engineering career path, helping individuals understand what is required to progress and succeed in their roles.

Mapping the expectations for these dimensions according to the level

You may ask why I didn’t put this in a table format instead. I believe that going for a narrative format makes it more accessible for a blog post or newsletter, and it is especially easier to consume from mobile. Ah, yes, and Substack doesn’t support tables.

Below, I describe the expectations or calibration for every dimension at every level of a software development career. This can be used for hiring, performance reviews, promotions, etc. A specific individual can be differently calibrated in different dimensions; for example, it is normal to see employees on track for promotion who perform consistently at the next level for specific dimensions but still need to work on other dimensions and in the opposite direction: some individuals can be strong in their role but have specific weaknesses or growth areas in some of these dimension that don’t affect the overall ability to be competent in the position.

Let’s cut to the chase:

Entry-Level Engineer

• Problem-Solving and Complexity: Solves well-defined problems with guidance and handles simple tasks.

• Autonomy and Execution: Requires significant oversight and mentorship. Executes tasks as directed, focusing on learning and growth.

• Technical Skills: Has a basic understanding of programming languages and tools, requiring guidance for most tasks.

• Leadership and Mentorship: No leadership role; focus on individual contribution.

• Impact: Impact is limited to individual tasks and projects.

• Communication: Communicates effectively with peers and supervisors.

• Ambiguity: Requires clear instructions to perform tasks.

• Process Improvement: Participates in team discussions about process improvements.

Mid-Level Engineer

• Problem-Solving and Complexity: Solves moderately complex problems with some guidance and handles moderately complex tasks.

• Autonomy and Execution: Works independently with occasional guidance. Executes moderately complex tasks independently.

• Technical Skills: Has a solid understanding of programming languages, tools, and frameworks and can work independently on most tasks.

• Leadership and Mentorship: May lead small projects or teams and guide entry-level engineers.

• Impact: Impact extends to team-level projects.

• Communication: Communicates well with team members and cross-functional teams.

• Ambiguity: Can handle some ambiguity with occasional guidance.

• Process Improvement: Suggests improvements based on observed inefficiencies.

Senior Engineer

• Problem-Solving and Complexity: Solves complex problems independently and manages complex tasks and projects.

• Autonomy and Execution: Works independently. Executes complex tasks effectively and provides guidance to others.

• Technical Skills: Expert in multiple programming languages, tools, and frameworks, and can design complex systems.

• Leadership and Mentorship: Leads significant projects or teams and mentors less senior engineers. Participates in hiring and promotions and provides relevant input for appraisals.

• Impact: Significant impact on projects and teams, driving key initiatives.

• Communication: Excellent communication skills, presenting complex ideas to diverse audiences.

• Ambiguity: Comfortable with ambiguity and finding clarity in complex situations. May mentor others in dealing with ambiguity.

• Process Improvement: Identifies areas for process improvement and implements solutions.

Principal Engineer

• Problem-Solving and Complexity: Identifies and solves highly complex problems and handles highly complex projects and systems.

• Autonomy and Execution: Operates autonomously. Ensures successful execution of large-scale initiatives and makes critical decisions impacting multiple teams.

• Technical Skills: Deep expertise in several areas, recognized for technical depth and breadth.

• Leadership and Mentorship: Leads large-scale initiatives, influence engineering direction, and mentors senior engineers.

• Impact: Drives organizational changes, impacting multiple teams and projects.

• Communication: Influences stakeholders across the organization and is effective at all levels.

• Ambiguity: Thrives in ambiguous situations, providing direction and solutions.

• Process Improvement: Leads initiatives to improve processes across teams.

Staff Engineer

• Problem-Solving and Complexity: Identifies and solves highly complex problems and manages highly complex tasks.

• Autonomy and Execution: Operates autonomously and drives execution of organizational-level projects, aligning execution with strategic goals.

• Technical Skills: Deep expertise in several areas, recognized for technical depth and breadth.

• Leadership and Mentorship: Leads large-scale initiatives, influence engineering direction, and mentors senior engineers.

• Impact: Drives organizational changes, impacting multiple teams and projects.

• Communication: Influences stakeholders across the organization and is effective at all levels.

• Ambiguity: Excels in highly ambiguous environments, setting clear paths forward.

• Process Improvement: Leads and drives process improvement initiatives across the organization.

Distinguished Engineer

• Problem-Solving and Complexity: Solves the most challenging problems in the field and addresses complex challenges.

• Autonomy and Execution: Operates with complete independence and executes at the highest level, ensuring strategic alignment.

• Technical Skills: World-class expertise in specific domains, recognized as an authority in the industry.

• Leadership and Mentorship: Shapes the company's technical strategy, influences industry standards, and mentors principal engineers.

• Impact: Impacts the entire organization and influences the industry.

• Communication: Communicates vision and strategy at the highest levels and influences industry discourse.

• Ambiguity: Masters ambiguity, defining new directions and influencing industry standards.

• Process Improvement: Defines and sets best practices and processes for the entire industry.

How to use all this?

Hiring and job descriptions

Use bits and pieces to compose your job description, being specific about what you are looking for and what the role's expectations will be. This will help you target the right candidates and have a clear reference framework for the specific kind of contribution and level of performance expected in the different areas. Then, during your hiring process, you should collect relevant evidence of how the candidate operates against those dimensions.

Performance reviews

What I like to do is create a matrix with all the dimensions, and for all of them, use collected evidence to rate the specific dimension (e.g., from 1 to 5) and add some examples to justify your position. You can assign different weights to different dimensions, or consider them equally important, it is up to you. At the end of the exercise, you will have a -reasonably- quantifiable way of putting it all together. One important aspect, and not easy to master, is not to compare employees against each other but against your previously defined calibration to reduce bias and avoid comparing heterogeneous situations.

Promotions

You can define a qualitative bar for the level based on the expectations per dimension and use a process similar to that described above to translate to a reasonable quantitative measure of how the employee compares with that bar. You can consider some dimensions mandatory and others, not a promotion blocker (i.e., something that can be learned or improved later in the first months in the role), and you can assign different weights to different dimensions. Collect examples that show consistent performance at the expected level and consistency in the expected behaviors, and include them as evidence of the readiness for the role.

Conclusion

If you are looking into your own professional growth and career path, or if you are a manager looking into hiring the right talent, or helping the career growth of your teams, understanding what different aspects are relevant, how a level compares with the others, and how specifically the growth is defined, this framework with help in these ways:

• You can evaluate and calibrate yourself and others with the right granularity and without ambiguity.

• The granular awareness helps focus the efforts on the areas that require investment, leveraging the strengths, and finding the right challenges and opportunities for intentional, accelerated growth.

I cannot avoid thinking about the Provisional Driving License or Learning Licenses that some countries use to help new drivers gain experience and skills safely before obtaining an unrestricted license. For a specific period of time, new drivers have specific restrictions, like speed limits, zero tolerance for alcohol, etc. After that set period of time, you stop being considered a newbie, and the restrictions don’t apply anymore because now you have some experience. A fixed period of time, it doesn’t matter whether you drove at all or not during that time.

That is, in a nutshell, what years of experience mean at work.

Time gives you nothing by itself.

In Spanish, a proverb says something like: “The devil knows more because he is old than because he is a devil” (Más sabe el diablo por viejo que por diablo). Well, that’s not true.

It is assumed that with age comes wisdom, but age only means time; it doesn’t say anything about what you did with that time. In the same way, space is a function of speed and time, and a lot of time passing doesn’t imply that you covered a lot of space; for that, you also need to assume some significant speed.

When we attribute expertise to a professional because of not being young or wisdom to elders just because of being elders, we are condescending: a smart, mentally ambitious person who is older than others and filled that time with continuous challenges and pushing boundaries, likely has a greater depth and breadth of knowledge and experience and is way ahead intellectually. But a person who got old without adding much to that depth and breath, without expanding perspectives and exploring new fields, without continuously challenging his own beliefs or skills, is just older and dumber, more encapsulated in the same ideas, with smaller and smaller horizons, and more corrupted by the endogamy of his or her own thoughts.

Time can take you to great lengths if you keep moving, but it will also help you rotten if you don’t.

How time can set you back

One of my favorite maxims is this fragment from Lewis Carroll’s Through the Looking-Glass (the sequel to Alice's Adventures in Wonderland):

- ‘Well, in our country,’ said Alice, still panting a little, ‘you’d generally get to somewhere else — if you ran very fast for a long time, as we’ve been doing.’

- ‘A slow sort of country!’ said the Queen. ‘Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!’

I think it describes perfectly the idea I’m trying to convey here.

You may be more or less smart and have more or less knowledge or experience at a certain point, but your ideas are discrete and not infinite. When you have a limited set of perspectives and do not continuously expand them over time, your thought processes become insular and self-reinforcing, leading to several negative cognitive and behavioral outcomes. Let’s talk about some of them:

Self-feeding or Circular Feeding of Thoughts

1. Echo Chamber Effect: When you are exposed only to ideas that reinforce your existing beliefs, you are less likely -and less interested- to encounter information that challenges or broadens your perspectives. This creates an echo chamber where the same ideas are repeated and amplified without critical examination. You just believe what you always believed, more and more.

2. Confirmation Bias: With limited ideas, you tend to seek out information that confirms your preconceptions, ignoring or dismissing contradictory evidence. This reinforces existing biases and makes it harder to accept new, potentially valuable information. Amazon has this leader principle “Are right a lot”, that says that leaders seek to disconfirm their believes.

3. Intellectual Stagnation: Without exposure to diverse ideas, your thinking can become stagnant. You are less likely to engage in creative problem-solving or innovative thinking because you are not challenged to consider alternative viewpoints or solutions. Your ideas will rot and smell like water in a pond.

Endogamic Nature of Thoughts

1. Narrow Perspectives: Endogamy in thinking refers to only engaging with ideas that come from within a limited circle. This can lead to a narrow worldview where you miss out on the rich diversity of experiences and knowledge available from different cultures, disciplines, and backgrounds.

2. Reduced Critical Thinking: When your ideas are not subjected to external scrutiny or diverse perspectives, you are less likely to develop strong critical thinking skills. Not only you think you are right, also your capacity of telling apart good from bad diminishes. Critical thinking thrives on evaluating different viewpoints and synthesizing information from various sources or at least taking them into consideration.

3. Increased Bias and Prejudice: A lack of exposure to diverse ideas can lead to the development of strong biases and prejudices. Without understanding different perspectives, you may develop misconceptions and stereotypes about people or ideas that are unfamiliar to you.

Impact on Awareness of Multiple Realities and Complexities

1. Simplistic Understanding: The world is complex, and understanding it requires acknowledging and integrating multiple realities and perspectives. When your ideas are limited, you tend to oversimplify complex issues, missing out on the nuances that are crucial for a deeper understanding.

2. Inflexibility in Thinking: Limited ideas can make you inflexible in your thinking, as you are not accustomed to adapting your viewpoints in light of new information. This rigidity can prevent you from effectively navigating and responding to the complexities of the world.

3. Reduced Empathy: Understanding and empathizing with others often requires recognizing their unique experiences and perspectives. A narrow set of ideas can limit your ability to relate to and understand people who are different from you, reducing your overall empathy.

And what if you were quite smart?

I hate to break this to you: it is worst. That will multiply the speed of your downfall.

When you are dumb and time passes, you may stay as dumb or get dumber, and you are likely aware that you are not the smartest and probably you did never care for it.

But when you were smart once, or when you are a bit over the average in intelligence or experience, you have something against you: you have a certain level of self-confidence and entitlement that will mislead you. I think about it as the opposite of the impostor syndrome. You were smart once, and that makes you think you still are, while the truth is that maybe others had years to get ahead of you, and you didn’t realize at what point you got behind.

I believe this is one of the reasons leading to the Dunning-Kruger effect, a cognitive bias that causes people to overestimate their own abilities and competence due to a lack of self-awareness and knowledge. Those experiencing it have excessive self-confidence and may not recognize their own incompetence, leading them to believe they are more capable than they actually are. This overconfidence can result in poor decision-making and resistance to feedback or learning opportunities.

Key characteristics of the Dunning-Kruger effect include:

1. Overestimation of abilities: They believe they are more skilled or knowledgeable than they truly are.

2. Lack of self-awareness: They are often unaware of their limitations and deficiencies.

3. Resistance to feedback: They may dismiss constructive criticism or advice from others, believing they know better.

4. Poor decision-making: Their overconfidence can lead to flawed judgments and decisions.

5. Difficulty recognizing expertise: They may have trouble recognizing true expertise in others and overvalue their own opinions.

In contrast to impostor syndrome, where individuals doubt their capabilities despite evidence of their competence, the Dunning-Kruger effect involves an inflated self-assessment that is not supported by actual skills or knowledge, creating a vicious cycle which will sequentially exacerbate the situation, leading to a worsening state over time.

If you were smart but you didn’t do much about it…most probably, after some time, you will be behind those that were just average or even mediocre.

Hic sunt leones

This Latin saying, that translates to "Here be lions", was historically used by cartographers on maps to indicate unexplored or dangerous territories, symbolizing the unknown or potentially perilous nature of those areas. The idea is that beyond the known world, there could be lions or other dangers lurking.

Unknown unknowns.

When I was a senior developer, with some 10 years of very deep experience and working quite actively in system design and solution design, I remember coming across a job offer for a Senior Software Architect. I wondered -I honestly couldn’t figure it out- what an architect can possibly do that I was not already doing.

Then time passed and I learned a few more things that I didn’t know and made me understand that there were so much that I didn’t know before, and then one more thing, and one more…so I understood there would always be much more I cannot even figure out at any given stage.

It is some kind of magic when one day you realize that you are in front of something you didn’t know: a previously unknown unknown becoming a “known unknown”, and your world becomes a bit bigger, your horizons expand a bit, and you are conquering yet one more step ahead of yourself, adding one more dimension. Like clouds opening and revealing the sun behind them.

One of the most exciting things in life is the certainty that there is always something else to learn, something else that you cannot even figure out now because it is not a bit on top of something that you already know but something that you don’t even know yet that exists. A world of knowledge beyond the known world.

Squeezing time to be at the top of your game

As the Queen of Wonderland said, you will need to run as twice as fast to go anywhere. Times pass whether or not you move. And everything around you is moving.

Expanding your horizons and challenging your own thinking will lead to greater intellectual growth and a more comprehensive understanding of the world. There are a few simple pieces of advice I can share (this is intentionally an unordered list; the order says nothing about their relative importance):

• Embrace lifelong learning and curiosity. Learn continuously and be open to new experiences and knowledge. The more you expose yourself to diverse ideas, the more you will be able to understand and navigate the complexities of the world and find more and more things that interest you, give you curiosity or reveal something that was obscure or unknown to you.

• Seek diverse perspectives: Actively seek out and engage with ideas that are different from your own. Read widely, talk to people from different backgrounds, and challenge your own assumptions regularly.

• Critical reflection: Develop the habit of critically reflecting on your own beliefs and the information you consume or produce. Ask yourself why you hold certain beliefs and whether there is evidence that contradicts them. Especially when you are too sure of something, take a step back and try to prove yourself wrong.

• Leverage negative feedback loops. This is not only about other people giving you feedback. There is more to it. Feedback loops are a fundamental concept across various fields of knowledge in which there is a fit for systems theory and dynamic processes. They help maintain stability in systems with mechanisms that detect deviations and trigger responses to correct them (for example, body temperature regulation in biology, supply and demand equilibrium in economics, predator-prey population dynamics in ecology, etc). In human behavior and social systems, feedback loops are the most basic and most powerful mechanisms in behavior regulation, social dynamics, and organizational behavior. This is by embracing and actioning the feedback that others give you, but also the feedback that the world gives you: a failed interview, a failed exam, a romantic rejection, finding out you were wrong about something about which you were very assertive, etc. There is always so much to learn if you take the time to absorb it and reflect on it. I have a separate article about leveraging feedback:
  

• Time is a fractal space; there is time inside the time. There are days inside the weeks and hours inside the days. How many hours do you work every week? When you like something, when you want to get ahead on something, the day doesn’t have just 8 hours. There are people who work all day, who work on weekends, who work on vacation, and it doesn’t feel like work. If a person works 80h per week and other work 40h…in the number same years not only there is double factual experience…more importantly: experience and growth are not linear functions, but maybe something like quadratic or exponential. Being everything else the same, reaching someone who is already ahead will be a very, very hard thing to do.

  To explain this visually, imagine the orange line is the person working double hours compared to the person represented by the blue line. The X axis is the time, and the Y axis is the relevant experience if we could quantify that as a number. The orange gets ahead but a significant amount.

  

• Choose your battles, and let them be a lot. You need to accumulate significant, relevant, and broad experience, not just experience. What specifically you have done matters as much as how much of it you have done. If you are a javascript developer that just did web forms 10 years for 80h per week, probably you won’t get ahead of someone who worked less years and less hours in a significant broad spectrum of technologies and domains.

A bit of my own story

I think about my career, and it could have been hard to predict. At different points in time, I was hands-on in qBasic, Pascal, C/C++, Java, PHP, Android, Angular, NodeJS, Python, and a longer list. I did video games, embedded systems, desktop applications, websites, and mobile applications. I made music and content and marketing campaigns and analytics, Facebook apps and WordPress sites, streaming systems, distributed systems, ERPs and CMS and integrations and ML, and white label multi-tenancy systems. I did system design, infrastructure, architecture, and observability, and I created vision, roadmaps, and PnLs. Most of the things that my teams have to do now…I have done it at some points with my own hands. That gives me a sweet yet serious perspective, some respect for the people who do the work, and some ability to give certain directions. Most importantly, most things don’t arrive as a surprise, and I can speak comfortably.

Had I spent my life doing always the same thing, my experience could be long, but I would miss the depth, breath, and my experience wouldn’t be intricate and intertwined. And that’s why makes me feel experienced and mature, not the fact alone of having been around for some time.

That’s why I value the multiple dimensions of the experience, how many different battles you fought and more or less survived for some period of time.

Not time alone, but you do need time

Said all this, time alone doesn’t give you experience by itself, but you do need time to be able to obtain it. All the above doesn’t happen overnight, and I reject the idea of someone being so good that can skip the line and go straight to the top without the time and the opportunities to be exposed to a number of situations and dedicate some sufficient time to it.

Conclusion

Time alone won't take you anywhere, and it can even take you back because your knowledge rots if you don’t keep learning and growing every day.

There is always a whole world ahead of you that you don’t know just yet, and it takes you to be intentional to set foot on it and time to explore.

Age and time don’t mean experience, and they don’t mean wisdom. Time gives you additional white pages, but it is up to you to write something meaningful on those.

This is especially true for unsolicited feedback. But it is also true that getting feedback at the right time is as rare as finding those sensors in an unsophisticated rental car you chose just by sorting by price.

Why wouldn’t someone give feedback?

There are a few reasons why people don’t give feedback when there would be a fit. In my view, mainly these:

• They don’t care about people, neither personally nor professionally. To even be bothered, you need to care to some extent about the specific person and about the whole concept of people and people dynamics.

• They are unable to tell the good from the bad or see enough steps ahead of situations or behaviors to realize there is room for course correction or for doubling down on something.

• They don’t know how to face people, be candid but not offensive, or respond to possible reactions, and they are unwilling to make the effort.

I don’t particularly enjoy working with people who check the boxes above (especially the first two). I think it is dysfunctional when those people are in leadership, and poor you if you happen to be reporting to one: far from helping you grow, they will slow down your career and your personal growth, and if you are self-aware enough, your motivation and energy as well.

Being able to compose, elaborate, articulate, and deliver valid feedback is not a nice-to-have ability but one of the foundational grounds of leadership. If you are a leader, or you think you are, or you want to be, you better get good at it.

On the other hand, of course, it also takes a good receiver. And yes, you better be a good receiver.

On receiving feedback

Very recently, someone I genuinely respect gave me a good piece of feedback and started the sentence with something like, “Sometimes feedback is necessary, I received feedback many times in my career and…etc.”. It was nice because probably this person didn’t know me enough and thought he may need to massage me or pave the way before throwing the bomb, but I was thinking, in a funny way, oh yes, clearly you don’t know me enough, because I treasure feedback, and I receive feedback as Wilt Chamberlain would control the basketball off the backboard (sorry, that means well).

I was not always like this, but I learned to receive feedback smoothly, painlessly, and fruitfully in the last few years. Receiving feedback well includes separate dimensions:

• How is that moment: awkward, violent, smooth, funny, interactive or one way, etc.

• What happens later: was it all for nothing, or did something come out of it? Did anything really need to come out of it?

Let’s go one step at a time.

The ball-catching moment

Following with my sports metaphors, the feedback sent to you is like a baseball when you are the catcher,

• it can hit you right in the face

• You can try or pretend to catch it but yet fail miserably and see it bouncing, maybe violently, maybe ridiculously, away from you

• it can totally miss you and land awkwardly nowhere while you seem to be oblivious, looking at something else

• or you can grab it gracefully with your hands

The ball thrown to you exists, regardless of how you deal with it. So you will be definitely better off if you catch it gracefully so you can deal with it. If you receive feedback, listen actively. Or, to put it simply, if you receive feedback, receive it.

The comeback, or how you react

How you react to feedback has much to do with the specific feedback but much more with your personal traits. Most people have a personal way of reacting to feedback, regardless of what the particular feedback is.

Personally, receiving feedback is something I like because:

• When it is something I didn’t expect at all, it makes me curious. Is there something I didn’t see? Did more people see the same? Is there something that is not like that but looks like that from the outside?

• If it is something I expected or was suspicious about, it is a relief. It is like a secret being revealed, a puzzle being solved. I knew there was something; finally, I know what it is. Now I can play knowing the cards.

You may have similar or different perceptions. But if you appreciate feedback, 99% of it is done.

On the other hand, If you are of that kind that feels offended or distressed or becomes defensive or angry when you receive feedback, there are typically two main reasons for it:

• that you think the feedback is totally off base,

• or that you think it is totally correct.

Reacting to untrue feedback is one kind of art. My advice is to listen politely, reply politely, ask for details, and take them home.

If you get angry and defensive because the feedback is correct, that’s easier said than done, but just don’t. You have been caught and don’t like it, but you cannot get uncaught by denying it. It is like playing hide-and-seek and covering your eyes with your hands when caught: you can choose not to see the reality, but others still do. My advice here is, again, listen politely, reply politely, ask for details, and take them home.

Extra mile tip: The best way to get details that can help is by asking follow-up questions, probing, and diving deep. Like you would do in a job interview in which you are the interviewer. You want to get to the bottom of it: why do you think that, what did you do or didn’t do exactly, what moment was key, what do you think should have been done differently, etc. Recollect all possible details. All the pieces matter.

The takeaway

Making sense of feedback in the moment of receiving it is a mistake. It is not possible. You must step back, look at things with perspective, process, and digest them.

That’s why the work starts later: take the feedback home.

My advice here is simple:

• Ask yourself sincerely about it; maybe you will find something about it to be true. Work actively to try to disconfirm your own beliefs and play devil’s advocate against yourself. Perhaps you find something to be true. Or maybe not, in that case:

• Keep track of things people tell you. If more than one person tells you the same thing over a period of time, chances are that it is true and everyone can see it but you, or very similar: that is not true, but it is how it looks from the outside. Perception is more important than reality because, like it or not, you will be evaluated for what is perceived of you, not for whatever happens only inside of your mind.

• If you find something, go to the bottom and understand the root cause and what you can do differently.

The aftermath

What happens in the short and long term with all this is what ultimately matters.

It is annoying and frustrating when someone is called out or given feedback, seems to understand it, but soon repeats the same wrong behavior. People just lose hope.

See this example below:

Sarah, a software developer, has repeatedly committed code without running the required unit tests. Her manager, John, noticed this and provided feedback.

John: “Sarah, I noticed you've been committing code without running the unit tests. It's important because..."

Sarah: "I understand. I'll make sure to run the tests before committing my code from now on."

A week later, John checks the latest commits and finds that Sarah has again committed code without running the unit tests.

John: "Sarah, we discussed the importance of running unit tests before committing code. Yet, it seems like this hasn't been done again. Can you tell me what happened?"

Sarah: "Oh, I'm sorry. I was hurrying to meet the deadline and thought it wouldn't be an issue just this once."

Sarah initially seemed to understand and agree with the feedback but soon reverted to the same behavior, indicating that the feedback was not effectively internalized or prioritized in her actions.

If you ignore feedback or receive it but don’t act on it in the short and long term, your manager, colleagues, and whoever is around you will lose trust and hope, feel not taken seriously, and doubt your actual abilities. They will feel the effort of the feedback was a waste of time, honesty, and trust, and you will come across as someone unwilling to take the extended hand. You will transform yourself into an incompetent buffoon.

To avoid this, extract learnings and apply them:

• Identify what are the key points that have to change

• Decide how exactly you have to do it. Sometimes, it is not so easy:

  • In cases of positive actions, i.e., things that you should start doing, you may want to transform the good intentions into mechanisms. For example, if you always forget to implement and run test cases, maybe you should add a subtask in your story to do so. It would be impossible to miss if it was part of a checklist. You can influence your team to do the same so you transform your problem into an improvement for yourself and everyone else.

  • In case of negative actions, such as “don’t do this: this” or “things you need to do differently, you may need to strategize more. For example, if you receive feedback that you complain a lot, clearly, you need to complain less. Still, you want your message to come across so you won’t simply shut up moving forward. Hence, what you need to reflect on is what words you are using that are perceived negatively and how you can transform your wording to transform a negative message into a positive one. For example, instead of saying:
    
    “Richard is useless and is making everyone go slower. We will never deliver the project.”

    You can say:
    
    “Things are challenging at the moment, but we are working around it: I’m trying to coach and support Richard, who, as you know, is relatively junior but yet a key player in the project. So I’m trying to reach where he cannot and give him direction, but this seems to be impacting our velocity. After the sprint review, I will let you know once we have more quantitative data, and we can see if there is an actual material risk.”

• Try to find opportunities to implement the changes. Listen actively and think before saying anything: “Could this situation or what I would say apply to the feedback recently received?” In that case, apply what you have learned and make sure you openly observe how it works.

This will show early evidence that the feedback is received and change is possible. It will also allow you to field-test your improvement strategy and iterate if needed.

That's a bummer, because sometimes your post is long enough to require some sort of index or to offer some idea of the content, so you end up creating a table of content manually (for example in this way).

Doing a TOC copying links and text from the actual post is quite mechanical work, and everything that is mechanical work seems like something you can write a script for. I put together that thought, having some time off, and that i don’t write a line of code in several months (or maybe more), and I ended up creating a script to generate a very simple table of content.

You can see an example in my post ChatGPT basics:

Usage

• make sure you have python or learn here how to get started

• Install lxml with `sudo pip3 install lxml` or check instructions here

• download my script and save it to any folder you want

• execute it with:

python3 substack_toc.py URL [output_filename]

# URL is the full url of the post 
# output_filename (optional). Filename in which the generated content will be saved. A file with that name will be generated in the folder in which you are running the script. It will be overwritten if it exists. If you don't include this parameter the content will be shown in the terminal.

• open the output file in your browser

• copy the content, and paste it in your substack post!

The script

You can find the script in GitHub Gist or here:

import requests
#you can install lxml with `sudo pip3 install lxml`
from lxml.html import fromstring
import sys
from urllib.parse import urlparse, urlunparse, urlencode, parse_qs
import time

if len(sys.argv) < 2:
   print("No arguments were given. Use URL [output_file_name]")
   quit()

#To avoid cache we add a parameter withe the timestamp
def add_timestamp_to_url(url):
    parsed_url = urlparse(url)
    query_params = parse_qs(parsed_url.query) | {"timestamp": int(time.time())}
    return urlunparse(parsed_url._replace(query=urlencode(query_params, doseq=True)))
    
url = sys.argv[1]
filename = sys.argv[2] if len(sys.argv) > 2 else None

#fetch the HTML
tree = fromstring(requests.get(str(add_timestamp_to_url(url))).content)

path = "//*[@class='header-with-anchor-widget']"
current_level = 100
ul_open = 0

output = "<ul>"
#get all headers with the right class define above
for header in tree.xpath(path):
	header_level = int(header.tag[1])
	print("H"+str(header_level) +" - " + str(header.text))
	
	#we will nest subheaders inside of parents
	if header_level > current_level:
		print("nesting ")
		output = output[:-5] + "<ul>"
		ul_open = ul_open + 1
	#close current tree and go back to a higher rank header
	elif header_level < current_level and ul_open > 0:
		while (ul_open > 0):
			print("unnesting " + str(ul_open))
			ul_open = ul_open -1
			output = output + "</ul></li>"

	current_level = header_level
	
	#create link
	link = header[0].get('id')
	output = output + "<li><a href='"+url+"#"+str(link)+"'>"
	output = output + str(header.text) + "</a></li>"

#if this is the last one, close current tree before closing the main UL
while (ul_open > 0):
	print("unnesting " + str(ul_open))
	ul_open = ul_open -1
	output = output + "</ul></li>"

output = output + "</ul>"

if filename:
	with open(filename, 'w') as file:
		file.write (output)
		print ("\nSaved to ./" + filename)
else:
	print(output)

Transformers are a type of model used in machine learning, particularly for tasks that involve language, like translating between languages, summarizing text, or changing the tone of text. They are called “transformers” because they can transform one form of data into another.

How Do Transformers Work?

Imagine you want to rewrite a sentence from a formal tone to a casual tone. Here’s how a transformer helps in doing that:

1. Input Processing: The transformer takes in the sentence and breaks it down into words or small parts of words (like “The president” becomes “The” and “president”).

2. Attention Mechanism: This is where it gets interesting. The transformer looks at each word in the sentence and decides which other words are important for understanding its meaning. For example, in the sentence “The president announced a new policy,” the word “president” might pay more attention to “announced” and “policy” because those words are key to understanding what the president did.

3. Understanding Context: The attention mechanism allows the transformer to understand the context of each word, not just its individual meaning. This helps it capture more nuanced meanings which depend on other words around them.

4. Generating Output: Based on this understanding, the transformer can then rewrite the sentence in a new tone while maintaining its original meaning. It does this by predicting words that fit the new tone but still convey the same information.

Encoding “Tone” in Transformers

Understanding how something like “tone” is encoded and managed in language models involves delving a bit deeper into the underlying mechanisms of transformers and their training data.

1. Data and Annotation: First, to handle “tone”, a transformer model needs training on a dataset where text is not only available but also annotated with tone information. This could include labels such as “formal”, “casual”, “sarcastic”, etc. Each piece of text in the training data must be tagged appropriately so the model learns what patterns correspond to which tones.

2. Embeddings: At the heart of how transformers process text are “embeddings”. Embeddings are vector representations (lists of numbers) of words that capture their meanings, relationships, and properties like tone. These embeddings are learned from the training data. When a transformer model is trained, it adjusts these embeddings to encode not just the literal meaning of the words but also nuances such as tone based on the context they appear in.

3. Attention and Contextual Understanding: During training, the attention mechanism of the transformer learns to pay more or less attention to different words in a sentence based on their contribution to the meaning and tone. For example, in a formal tone, the model learns to focus on and prioritize words and structures that convey formality.

4. Storing Tone Information: Technically, tone is not “stored” in a specific location but is rather encoded throughout the network in the weights and biases of the model—these are the parameters that determine how input signals are transformed through the network layers. The way words and phrases relate to each other in terms of tone becomes part of the overall parameters of the model.

Associating Tone with Words and Sentences

When a transformer model processes a sentence, it uses both the learned embeddings and the contextual clues picked up by the attention mechanisms to infer the tone and adjust the output accordingly. Here’s how it works step-by-step:

1. Input Analysis: The model reads the input sentence and maps each word to its corresponding embedding.

2. Attention Processing: As the model processes the sentence, its attention mechanisms determine how the meanings and tones of words influence each other. For example, the presence of words like “please” and “advise” in a formal context might enhance the formal tone through the attention they receive relative to other words.

3. Output Generation: When generating text, the model uses its understanding of both the literal content and the tone to select words that match the desired output tone. If the desired tone is “casual”, the model looks for words and structures in its training data that have been associated with a casual tone and constructs the output accordingly.

Example of Tone Adjustment

If a transformer is tasked to change the sentence “I request you submit your report by Monday.” to a more casual tone, it might generate “Hey, can you get your report in by Monday?” Here, it uses its training on how formal and casual tones are structured differently, choosing “Hey” and “get your report in” to replace the more formal “request” and “submit.”

One more example:

Original Text: “Please advise on the status of the current initiative as soon as feasible.”

Current tone: Formal and direct.

Desired Tone: Casual and friendly.

Transformed Text: “Could you let me know what’s up with our current project whenever you get a chance?”

Summary

This capability of transformers to handle such nuanced tasks as tone adjustment comes from their extensive training on large datasets, sophisticated attention mechanisms, and the ability to learn and manipulate embeddings effectively.

In this process:

• Transformers handle the heavy lifting of understanding and generating text.

• Attention mechanisms determine which parts of the input are important and how they relate to each other to better preserve the meaning across transformations.

You can go a bit deeper in this post:

Im very skeptic about the concept of improving. I have seen people improving furniture, their home, their clothes, or their cars, to make it considerably worst - in my opinion thought. The concepts of good or bad are subjective in many realms, thus the concept of improving is as well subjective.

When a text is fairly well written, you have decent grammar skills, and you are a fairly good communicator, chances are that your text is at least ok, probably good. And on top of it, it is yours, so it has the value of the unique style and genuineness. Is changing it an improvement? or just that, a change, changing it to a different version more generic, more standard, more flattened out? Maybe.

But ok, if your text is not so good, maybe the above can be an actual improvement.

But in both cases, how does a large language module qualify some text to be better than other. I decided to stop speculating, and go to the source, and I asked ChatGPT the same.

When I ask you to improve a text, how do you discriminative what is "improving", or how something can be qualitative considered better than something else?

What ChatGPT considers as improving a text

This is what ChatGPT told me:

When improving a text, I consider several factors to determine what makes one version qualitatively better than another. Here are the key aspects I focus on:

1. Clarity and Precision: Making the text clearer and more precise, ensuring that the message is understandable and unambiguous. This often involves rephrasing sentences, choosing more specific words, or reordering information for better flow.

2. Conciseness: Removing redundant words or phrases, and streamlining the text to make it more concise without losing essential information. This helps in making the text more impactful and easier to read.

3. Grammar and Syntax: Correcting grammatical errors and improving sentence structure. Proper grammar and syntax help in communicating the message effectively and professionally.

4. Style and Tone: Adjusting the style and tone to suit the intended audience and purpose. This could mean making the text more formal or informal, adjusting the level of technicality, or making the tone more persuasive, informative, or friendly, depending on the context.

5. Engagement and Interest: Enhancing the text to make it more engaging and interesting to read. This could involve adding rhetorical devices, varying sentence lengths, or incorporating compelling examples.

6. Coherence and Cohesion: Ensuring the text is logically organized and the ideas flow smoothly from one to the next. This may involve adding transition words, breaking up long paragraphs, or restructuring sections for better coherence.

Each improvement is aimed at making the text more effective for its intended purpose and audience. When suggesting changes, I balance these aspects to enhance the overall quality of the text.

Ok, the categories make sense. But isn’t it kind of a “circular reasoning” in a definitional context?. You say you improve it for example by doing it more concise, but same as I ask you how do you know it is improved, i can ask you how do you know it is more concise. I don’t like feeling trapped in a logical fallacy, so I have to go deeper. I imagine this can come from a semi-supervised learning, in which humans are labeling texts in regards of the dimensions above, and others, so then the model can learn from the context. So I did a bit of research, and a bit of prompting, and below is what i found -spoiler alert, my idea was wrong-

Reference data

GPT doesn’t require previously labeled data to improve text or to evaluate elements like clarity, conciseness, or engagement. Instead, its training involves learning from a vast array of text data and editorial examples, which has given it an understanding of language norms, styles, and effective communication strategies across various contexts.

Here’s how it approaches evaluating and improving text:

1. Language Models: Its core is based on large-scale language models trained on diverse datasets encompassing a wide range of topics, styles, and formats. This training helps it understand what constitutes clear, concise, engaging, and grammatically correct language.

2. Patterns and Probabilities: It analyzes text based on learned patterns and probabilities. For instance, certain phrasing might commonly appear in well-written articles, suggesting that it’s clear or engaging. Similarly, certain structures or word choices might be flagged as unclear or verbose in less effective texts.

3. Contextual Understanding: It uses contextual cues to adjust tone, style, and formality. The intended audience, purpose of the text, and the medium through which the text is communicated are all considered. This contextual understanding helps tailor improvements to the specific needs of the text.

4. Heuristics and Rules: Based on conventional rules of grammar and style, it applies heuristics for correcting errors and improving syntax. This includes adhering to grammatical norms and style guidelines that are widely accepted in various language use contexts.

5. Feedback Loops: While it doesn’t learn dynamically in real-time from user interactions due to its design as a static model, the iterative improvements and updates to AI models like it are informed by user feedback and advancements in natural language processing research.

This combination of learned data, context analysis, and established linguistic rules allows it to make educated judgments on subjective aspects of text quality, aiming to enhance readability, effectiveness, and audience engagement.

But again, when we say “a phrase might commonly appear in well written articles”, how do it tell apart in its training data which texts are well written and which ones are not?

The distinction between well-written and poorly written texts isn’t explicitly labeled by human reviewers for each individual example. Instead, the model learns to identify quality based on several indirect factors:

1. Source Quality: Texts are sourced from a wide variety of contexts, including reputable publications, educational materials, and other high-quality sources known for their editorial standards. The model implicitly learns from the quality of writing these sources typically uphold.

2. Contextual and Comparative Learning: Through exposure to vast amounts of text, the model develops an ability to compare and recognize patterns that distinguish clearer, more effective writing from less clear or error-prone writing. It notices how certain phrases and constructions are used across different texts and starts to associate specific structures and vocabulary with more polished, professional writing.

3. Corrections and Revisions: Training data can include edited versions of texts, where revisions made by editors or writers improve clarity, grammar, or style. By analyzing pre- and post-edit versions of texts, the model learns which changes enhance the quality of writing.

4. Feedback Mechanisms: Although it doesn’t learn in real-time, overall model improvements are made by incorporating feedback on the performance of earlier models. This includes adjustments based on how well the model’s outputs align with human judgments of quality in controlled testing and evaluation scenarios.

These methods allow the model to develop a nuanced understanding of what makes text effective, engaging, and clear, even without explicit labels distinguishing every example of good or bad writing during training.

Conclusion

LLMs assess and improves text quality, in terms of Coherence and Cohesion, Engagement and Interest, Style and Tone, Grammar and Syntax, Conciseness, Clarity and Precision, through a sophisticated integration of data, contextual analysis, and linguistic rules.

The model doesn't rely on explicit labels distinguishing well-written from poorly written texts. Instead, it learns to recognize quality through indirect measures, such as the inherent standards of texts sourced from reputable publications and educational materials. Through exposure to vast amounts of varied texts, the model develops a capability to identify and differentiate patterns indicative of clear and effective writing from those that are not. It also learns from revisions in texts, understanding which changes enhance clarity and correctness. Additionally, the model adapts and evolves based on feedback, aligning its output more closely with human judgments of textual quality.

This approach allows the AI to cultivate a deep understanding of what makes text engaging, effective, and clear, thereby improving its ability to enhance readability and audience engagement without direct guidance on every instance of text it analyzes.

Is not magic, is science.

• Capabilities of ChatGPT for Business

• Using ChatGPT in Daily Life

• Limitations

  • Search Engines vs Large Language Models

• Introduction to prompt design and prompt engineering

• How does it work?

  • How is it so different from previous ChatBots?

  • Attention is all you need!

• Alternatives

• Legal and Ethical concerns

ChatGPT: bridging the gap between human creativity and AI efficiency

We all know what ChatGPT is, an AI-based tool designed to understand and generate human-like text based on instructions it receives, with the aim of augmenting human capabilities in various tasks, such as customer service, content creation, programming, improving productivity in different fields, and fostering innovation by allowing users to explore new ways of creating with AI, offering benefits such as high-quality text generation, versatility in applications, time-saving for repetitive tasks.

But... what else is there to know?

Capabilities of ChatGPT for Business

Imagine having a highly skilled assistant who is always ready to help with your writing tasks, research, and even coding questions, without getting tired. That’s what ChatGPT offers to your business.

• Writing and Email Assistance: ChatGPT can help you draft a professional and polite message in no time, chosing the tone you want, and even when you are not sure about what you want to say.

• Proofreading and Editing: Enhancing written materials. ChatGPT can also play the role of a second pair of eyes, helping you spot errors or suggesting better ways to phrase your sentences, making your messages clearer and more impactful. It considers aspects such as:

  • Clarity and Precision

  • Conciseness

  • Grammar and Syntax

  • Style and Tone

  • Engagement and Interest

  • Coherence and Cohesion

• Surveys and Quizzes Generation.

• Market Research: Summarizing findings, trends, articles, reports, and social media buzz, giving you a concise overview of your industry landscape.

• Translations: Multilingual support for global reach.

• Product Descriptions: Crafting compelling copy.

• Coding: Basic coding help and code review.

• Automating customer service inquiries.

• Generating reports and business insights, research competitors, seek help in deciding whether or not to invest in a company.

• Additional Uses: Encourage imagination: “The possibilities are nearly endless. Whether you need help organizing information, generating ideas for a marketing campaign, or even learning something new, ChatGPT is here to assist.”

Using ChatGPT in Daily Life.

Ever wished for a personal assistant to help with daily tasks, from planning meals to learning a new hobby? ChatGPT can be that assistant.

• Meal Prep: Imagine deciding what to cook, and ChatGPT suggests recipes based on what you have in your fridge, along with step-by-step instructions and nutritional advice.

• Pocket Coach: Need motivation or advice on staying fit? ChatGPT can suggest workouts, provide tips for mental wellness, and even help track your progress, acting as your personal coach.

• Craft Project Instructions: Looking to start a new craft project but don’t know where to begin? ChatGPT can offer ideas, detailed instructions, and even troubleshoot problems you might encounter along the way.

• Personal Learning: Whether it’s mastering a new language, understanding complex topics, or just satisfying your curiosity, ChatGPT can provide explanations, resources, and quizzes to make learning enjoyable and accessible.

• Learning new languages.

• Planning travel itineraries.

• Have fun!: Did you know that you can invert the roles and have ChatGPT asking you questions? try:
  
  Test my knowledge on ancient civilizations
  

• Can help you find a famous sentence or a word you cannot recall, just by giving some relevant context about what you are looking for.

💡 With ChatGPT, technology is not just for the tech-savvy. It’s a tool that everyone can use to make both work and daily life easier and more enjoyable. Think of it as having a smart, helpful friend, always ready to assist you in navigating both your professional and personal worlds.

Limitations

Chat GPT collects information, and when you ask something it returns what seems most likely. “Most likely” is not the only truth, just as the first google result is not the only answer, and not always the one you are looking for.

There are several risks associated with relying solely on summarized information from GPT or similar AI models, instead of evaluating different sources -i.e. through a search engine- Here are some key considerations:

1. Accuracy and Completeness: AI models like GPT summarize and generate responses based on patterns in data they were trained on. This can sometimes lead to inaccuracies or omitting critical nuances that might be captured through direct research from primary sources.

2. Bias and Perspective: AI models can inadvertently reflect biases present in their training data. Relying solely on AI for information could lead to a skewed understanding if these biases are not recognized and countered by checking diverse sources.

3. Recency of Information: Lack of real-time knowledge beyond its last update. GPT models are trained on a dataset that only includes information up to a certain point in time. They do not have access to or the ability to pull in the most current data or news, which can be critical depending on the topic.

4. Critical Thinking and Analysis: Evaluating different sources on a search engine encourages critical thinking as you compare perspectives, assess the credibility of each source, and synthesize information from various viewpoints. Relying on a single summarized source can limit this analytical process.

5. Depth of Understanding: While GPT can provide a broad overview, it may not be able to delve into the depth that specialized articles or studies offer. Detailed and technical topics might be oversimplified, missing crucial details.

6. Hallucinations. The content can be wrong altogether even when it may look correct. More on this later in the article.

7. Need for human oversight. This may be seen just as a consequence of all the above.

To mitigate these risks, it’s often beneficial to use GPT as a starting point for getting a general understanding or as a supplementary tool alongside traditional research methods. This approach allows you to leverage the strengths of AI in generating cohesive, understandable content while ensuring accuracy, depth, and breadth through direct source verification.

Hallucinations

In the context of Large Language Models (LLMs) like the one I am based on, hallucinations refer to instances where the model generates text that is factually incorrect or not grounded in reality. These hallucinations can occur due to various reasons, including biases in the training data, lack of context understanding, or errors in the model’s language generation process.

Here are a couple of known examples of hallucinations in LLMs:

1. Medical Misinformation: In the context of healthcare, LLMs have been observed to generate hallucinations that can potentially misinform patients or healthcare professionals. For example, a model might generate inaccurate medical advice or provide incorrect information about treatment options or symptoms of a particular condition.

2. False News: LLMs have also been known to generate false news articles or headlines, contributing to the spread of misinformation online. These hallucinated articles can resemble real news stories but contain fabricated information, leading to confusion and potential harm if believed and shared widely.

Addressing hallucinations in LLMs is crucial to ensure the reliability and trustworthiness of the generated output, especially in sensitive domains like healthcare or news reporting. Researchers are actively working on developing techniques to mitigate hallucinations and improve the overall quality and accuracy of LLM-generated text.

Search Engines vs Large Language Models

Chat GPT doesn't know anything that Google doesn't know, and it has a bunch of limitations and risks, so if you want information, it's better to use Google?

Choosing whether to use Google (or any other search engine) or ChatGPT depends largely on the nature of your query and what you’re looking to achieve with the information:

Use Google when:

1. Specific Facts or Research: You need precise, up-to-date information or statistics, like news, weather updates, or specific research studies.

2. Range of Sources: You want to see a wide range of opinions or sources on a topic.

3. Local Information: You’re looking for local businesses, services, or location-based data.

4. Complex Searches: You want to perform advanced searches using specific operators or filters to refine your search results.

Use ChatGPT when:

1. Conversational Understanding: You’re looking for a more interactive, conversational approach to explore topics or ideas, or you need explanations in simpler terms.

2. Synthesis of Ideas: You want a summary of a broad topic or need to combine information from various sources into a coherent narrative.

3. Creative Content: You need help with generating creative content, such as writing, brainstorming ideas, or artistic concepts.

4. Language Tasks: You need assistance with language-related tasks like translation, grammar correction, or writing improvement.

Why To Choose GPT Over Google

• Interactive and Iterative Learning: ChatGPT can help explain complex concepts in a step-by-step manner, making it useful for learning and understanding nuanced details.

• Personalization: It can tailor responses based on the conversation’s flow, providing a more personalized interaction.

• Language Processing: It’s particularly strong in generating human-like text, making it ideal for writing assistance, dialogue simulations, and more creative tasks.

Why To Choose Google Over GPT

• Accuracy and Timeliness: Google provides access to current information and can offer more reliable and precise data, which is crucial for research-based tasks.

• Comprehensive and Diverse Perspectives: It gives you access to a vast array of voices and perspectives, which can be important for forming a well-rounded view on controversial or debated topics.

In summary, the choice between Google and ChatGPT hinges on whether you value interaction and synthesis of information or if you need up-to-date, specific factual data sourced from a wide range of perspectives.

Introduction to prompt design and prompt engineering

These are 2 very interrelated concepts.

• Prompt Design refers to the process of creating clear, effective questions or statements that guide artificial intelligence or language models in generating useful, accurate responses. Think of it as asking someone in a clear way for exactly what you need.

• Prompt Engineering, is a more technical aspect. It involves optimizing these prompts to improve the quality, relevance, and accuracy of the AI’s responses. Think of it as using psycology to really understand how others think and obtain the curated response you are looking for.

Key Concepts

• Clarity: The prompt should be clear and specific. Ambiguity can lead the AI to misunderstand the request.

• Context: Providing the right amount of background information in a prompt helps the AI generate more accurate and relevant responses.

• Conciseness: While detail is important, being concise helps prevent overwhelming the AI with unnecessary information.

• Goal-Oriented: A good prompt has a clear objective. What do you want the AI to do? Generate text, answer a question, create an image?

Guidelines and Tips

1. Start Simple: Begin with straightforward prompts to see how the AI responds, then adjust as necessary.

2. Iterate and Refine: Don’t expect to get the perfect answer on the first try. It may take several iterations to refine your prompt to get the desired outcome.

3. Use Examples: When appropriate, provide examples in your prompt. This can guide the AI in understanding the format or type of response you’re looking for.

4. Be Specific: The more specific your prompt, the more likely you are to get a relevant response. However, balance this with the need to be concise.

5. Test Different Approaches: There’s often more than one way to design a prompt that achieves the same goal. Experimenting can reveal which is most effective.

6. Use Templates. Not only save time, also be more efficient if you get used to predefined structures that work for you for common tasks. As a general guideline you can use the frameworks below.

Prompt Frameworks

BAB

• Before

• After

• Bridge

Example

• We are nowhere to be seen on SEO rankings (BEFORE)

• Wish to be in the top 10 rankings on niche keywords (AFTER)

• Include the entire list of keywords targeted by our competitors (BRIDGE)

RTF

• Role

• Task

• Format

Example:

• You are a Facebook ad marketer (ROLE)

• Design a compelling Facebook ad campaign to promote a new line of fitness apparel for a sports brand (TASK)

• Create a storyboard outlining the sequence of ad creative, including story writing, visuals, and targeting strategy (FORMAT)

TAG

• Task

• Action

• Goal

Example:

• Elevate the performance of team members (TASK)

• Act as direct manager and assess the strengths and weaknesses of team members (ACTION)

• Improve the team performance so that the average user satisfaction score increases from 6 to 7.5 (GOAL)

CARE

• Context

• Action

• Result

• Example

Example:

• Launching a new line of sustainable clothing (CONTEXT)

• Help us create a targeted advertisement that emphasizes our environmental commitments (ACTION)

• With the desired outcome to increase awareness and sales (RESULT)

• A good example of a similar successful

Others

• Do-To:

  • Explain nostalgia to a kindergartener

  • Write a thank-you note to my interviewer

• Do-Context

  • Make up a story about Sharky, a tooth-brushing shark superhero

  • Plan a trip to experience Seoul like a local

How does it work?

It's not magic, it's science!

AI and ML are closer to statistics than to other disciplines. Probability is the key word.

There are three main stages: data collection and training, prompt interpretation, and generating responses.

1. Data Collection and Training

Collecting Training Data:

• ChatGPT is trained using a vast collection of text from the internet. This includes books, websites, articles, and other forms of written content. The goal is to expose ChatGPT to as much human language as possible, covering a wide range of topics, styles, and contexts.

Training Process:

• During training, ChatGPT learns to predict the next word in a sentence based on the words that come before it. It does this millions of times -processing up to 300 billion tokens-, adjusting its internal settings -up to 175 billion parameters- to get better at these predictions. This process is a bit like a highly advanced form of guesswork, where the model learns patterns, structures, and even the nuances of language.

Outcome:

• The result is a model that understands language to a significant degree and can generate coherent, contextually relevant text based on the great amount of patterns it has learned.

2. Prompt Interpretation

Receiving a Prompt:

• When you give ChatGPT a prompt, the model uses its training to understand what you’re asking. It analyzes the words and phrases you use to figure out the context and intent behind your question or statement. Whaever you are asking, it has already read before. But... exactly the same? Well, maybe not exactly, but ChatGPT pays a lot of attention... remember this word!

Context Understanding:

• The AI considers not just the immediate prompt but also the sequence of prompts and responses in a conversation. This allows it to maintain context and continuity, enabling more coherent and relevant interactions.

3. Generating Responses

Predicting Possible Responses:

• Based on the prompt’s analysis, ChatGPT generates a range of possible responses. It uses its understanding of language and context to predict what a suitable answer might look like, drawing on the patterns it learned during training.

Selecting the Best Response:

• From these possibilities, the model evaluates which response is most appropriate, coherent, and contextually relevant. This evaluation is based on its training and the objectives set during its development, such as being helpful, accurate, and safe.

Output:

• The selected response is then refined and presented as the final output. This process happens in seconds, making it seem like ChatGPT is responding in real-time.

How is it so different from previous ChatBots?

A bit of history.

The overview below shows how chatbots have evolved from simple, command-based systems to sophisticated tools capable of engaging in nuanced conversations:

Early Chatbots (1990s - Early 2000s):

• Based on predefined rules and logic.

• Users had to choose options or type exact keywords to get responses.

• Example: Automated customer service on websites.

Development of Natural Language Processing (Mid 2000s - Early 2010s):

• Chatbots began understanding variations in how people type and talk.

• Introduced “utterances” where users could input more natural language.

• Example: Early versions of Siri and simple conversational agents on messaging platforms.

Advancement with Machine Learning (2010s):

• Chatbots use large amounts of data to learn and predict responses.

• Begin to handle more complex conversations and tasks.

• Example: Google Assistant, which can set reminders and answer varied questions.

Introduction of Transformers and Attention Models (Late 2010s - Present):

• Newest type of technology in chatbots.

• Focuses on the context of entire conversations for better understanding and responses.

• Can maintain more coherent and extended interactions.

• Example: ChatGPT, which can generate human-like text and engage in detailed discussions.

More bellow about Transformers and Attention.

Attention is all you need!

Ashish Vaswani, affiliated with Google, is the lead author of the paper "Attention is All You Need", published in June 2017, which introduced the Transformer model and the mechanisms of attention, which has since become foundational in the field of natural language processing and beyond.

Transformers are a type of model used in machine learning, particularly for tasks that involve language, like translating between languages, summarizing text, or changing the tone of text. They are called “transformers” because they can transform one form of data into another. How do they work?

1. The transformer decomposes sentences into words

2. Attention mechanism: This is where it gets interesting. The transformer examines each word and decides which other words are important to understand its meaning. For example, in the sentence "The president announced a new policy", the word "president" might pay more attention to "announced" and "policy" because those words are key to understanding what the president did.

3. Understanding context: The attention mechanism allows the transformer to understand the context of each word, not just its individual meaning. This helps him or her to grasp more nuanced meanings that depend on other words around them.

I have created a different post about transformers:

And a bit more on it in

Legal and Ethical concerns

I am largely out of my expertise area here, so i wont mention much about this, but here is a line of explanation for some of the main legal and ethical concerns around the use of models such as ChatGPT:

Data Privacy

Ensuring user data is handled securely.

• It is critical to implement robust security and compliance measures to protect users' personal information and ensure that their interaction with the system is confidential and secure.

• The legal team together with the information security team is responsible for ensuring compliance with data protection laws such as GDPR in Europe or CCPA in California, implementing technical and organisational measures to protect user data.

• Misinformation: Mitigating the spread of inaccurate information.

  • The model should be designed and adjusted to minimise the generation of false or misleading content and promote fact-checking and accuracy in the answers provided.

  • Product development and AI research teams work on improving the accuracy of the model and its ability to handle truthful information. This includes fine-tuning algorithms and validating the information generated to minimise errors and misinformation

• Bias: Addressing inherent biases in the AI model.

  • It is crucial to conduct ongoing audits and train the model with diversified datasets to reduce inherent biases and ensure that responses are fair and equitable.

  • AI ethics, AI research and development teams are responsible for auditing and mitigating biases. This involves reviewing and improving training datasets and adjusting algorithms to reduce bias and ensure equitable responses.

• Intellectual Property: Understanding the ownership of AI-generated content.

  • Understanding and defining who owns the rights to AI-generated content is essential, in addition to respecting copyright laws when creating or using content based on or derived from copyrighted sources. Regarding intellectual property and the use of content generated by models such as ChatGPT (point 4), the situation may vary depending on the provider and specific policies.

  • Generally, OpenAI (the company behind ChatGPT) allows the use of model outputs for a wide range of applications, but with certain restrictions related to direct resale or misuse (such as the creation of misleading, defamatory or copyright-infringing information). Users are responsible for how they use the generated responses and must ensure that their use does not infringe copyright or trademark laws.

  • Compared to other competitors, most follow similar models where the generated output can be used freely at the user's own risk, but may differ in specific details about commercial re-use or the generation of derivative content. Some may require special licences for commercial use or impose additional restrictions on the type of content that can be generated or shared.

    It is important to review each provider's terms of service to fully understand the rights and responsibilities related to the use of AI-generated content.

I don't consider good and bad luck at opposite ends of the same spectrum, and I absolutely differentiate the realms they belong to.

Bad luck

I find it self-indulging to talk about bad luck at all. This thing didn't happen because I had bad luck. I should be rich by now, but I had bad luck. This is like pretending that the usual thing we can expect is that things go smoothly and the world serves our purpose, and anything that deviates from that is bad luck. Well, this is very far from reality. There is no outcome at all that you can expect from any multidimensional, complex situation, so when you start anything, especially if it requires discipline, preparation, and effort, you should assume that any of the multiple things that should be true for your outcome to occur, will fail to be true. So failing is not actually a strike or bad luck; it is the default. This is not a negative thought but rather the opposite: what you can call bad luck is just the bottom line: anything positive has to be built on top of the default - which is nothing.

Good luck

I am a grateful person, and I’m conscious (or want to be because it feels better) that a significant part of my success is due to luck. I have worked 80 hours per week since I can remember. And I do try to push myself at any opportunity I have. But many people do the same or more and don’t get the same outcome. Also, some people have not done anything of value and are in positions they don’t objectively deserve. So there is something there that is an external factor, and we can call it good luck. 

But some would argue that good luck is just relentlessly jumping to opportunities when they pass before you. But let’s distill that. It would take:

• Identifying those opportunities. Opportunities don’t come with a sign; they are not self-evident.

• Knowing what “jumping” means. What is the next step? What should I do to leverage that opportunity?

• Transform whatever it is into something actionable, execute it, and react accordingly to the outcome of every step.

• And consistently make more good decisions than bad ones.

Seizing opportunities seems to require a significant amount of conscious effort and certain skills. Therefore, when you observe someone who has achieved certain success and -in otherwise equal conditions- it seems mainly attributable to external factors—i.e., luck—it's possible that those external factors are possible only due to the internal attributes of that person.

But then, well, about those internal attributes…we can close the loop by saying that they are also there by luck. When a person keeps putting to use the correct attributes (it doesn’t matter if it is talent, or getting away and surviving through shame and pitty or others), or a person keeps choosing right -or serendipitously- at the right time and perseveres in the right way, consciously or not, it may be because that person has, maybe by chance, maybe from birth, the skills or traits that allow them to act in a certain way: the ultimate “luck” of the genes or the upbringing and any other thing that determines us to be one way or another. 

So is it everything chance and luck? Even what seems intentional and conscious and your own merit? Is everything random?

But luck or not, what is the difference?

One way or the other, it doesn’t provide any value to determine if luck is a factor in anything. Ultimately, the whole concept is more of a matter of naming or definitions because the actual variables to control in a given situation remain the same, and you cannot leave anything to luck. Luck serves to describe things in retrospect but nothing else.

It is true, though, that talking about luck removes responsibility, and that can be the actual use -or misuse- of it. If things depended on me, I somehow would need to understand what I have done right and keep doing more of so, and assume that it is also on me when I fail because there are no random external factors. On the other hand, if luck is a primary factor, then life is easier, I just need to cross my fingers and try to keep catching green lights or surfing the wave, so I hold limited responsibility for the success. I also have a straightforward and unquestionable reason to justify if something doesn't go as expected, so I hold limited responsibility for failures as well.

If we rule out luck, what you have to do with your own success is some kind of fuzzy function that factors in many variables: preparation, effort, perseverance, awareness, awakeness, and drive, and those variables may have or will have different weights, and those weights remain the same whatever are there other external factors or not.

Further reading

It's not necessarily related to my perspective, but a real masterpiece on luck is Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets by Nassim Nicholas Taleb, one of my favorite writers.

This book explores how we perceive and handle luck in life and business (focusing particularly on trading, where luck is often mistaken for skill). It is a landmark work essential for understanding random outcomes. Taleb succeeds in tackling three major intellectual issues: the problem of induction, survivorship biases, and our genetic unfitness to modern life, and highlights our tendency to overestimate causality and view the world as more predictable than it is.

During the last few years, I came to the realization -this won’t surprise anyone- that not everyone is in the position in society that they should be in. This was probably not so clear to me before when I believed that after societies moved away from simpler forms of nepotism, we could assume meritocracy to be embedded organically in any system and the foundation of any hierarchy system. You are probably in a position that makes sense in the overlap of what you want, what you deserve, and what you are capable of. But the more I grew myself into higher leadership positions, and the more I learned about how hierarchies work in the public and private spheres, the more I have seen individuals that are either further down from the position they should be in and others that are totally out of place in a high position that they are not able to defend, but that they still keep somehow. 

I reflected on this a lot in the last few years. And I find it interesting to see how we can distribute these people differently over different dimensions:

• Intentional vs. accidental

• Aware vs. unaware

• Competent vs. incompetent 

• Confident vs. insecure

• Entitled vs. humble

All those aspects overlap in different points to give us output metrics such as legit or not, deserved or not, fair or unfair, reprehensible or commendable.

I’m developing this line of thought to take us to the dichotomy of opportunity and competence, as the title suggests. However, to arrive there, I would like to go over some of the components of the fuzzy Venn diagram that we could draw around this.

Before going into it, let’s take one thing out of the way: the definition or value of luck. I talk about this in a separate article:

Luck or not, my next question is: are you at a place where your steps took you or where the wind took you?

Intentional vs. accidental

Intentional

Let's talk about Sylvana. Sylvana found her way up the corporate ladder and is currently in a position that, objectively, she wouldn’t belong to. This is because she doesn't have the right education, experience, or capacitation and didn’t go through the legitimate process to achieve it. But still, she is there. Connecting the dots backward, there is one thing that legitimizes her position: intent. She made a continuous, intentional effort.

Is it more valuable to be an excellent singer or to be mediocre but number one on the lists and sell-out stadiums? Is it better to produce an excellent wine or a mediocre one that is a best seller and exported worldwide? It depends on what you are after. But definitely, if you have intention, you make the required effort, and you find your way up to where you want to be, there is something commendable about it. 

Sylvana is good at reading the room, finding alliances, influencing, and faking his way around people with enough power to make a difference. Also, she has a couple of skills that can be used as a facade to pretend a broader competence, communicates well, and can calculate meticulously every step. Also understands politics and has a Machiavellian approach to leadership.

We can talk forever about manipulation, deceit, and the moral cost of doing this business, but there is intent, there is effort, and there is a very well-oiled political mechanism. And the intentional, consistent effort worked out. That people overtake from the right. Maybe you were busy doing actual work, while others were busy just working on their position. When you see that person becoming successful, maybe at your own cost, you can hate it, but you understand what happened and even decide there is something you want to learn from it.

Accidental

On the other hand, Felix Luckster's story is quite different. He applied for a position, and the recruitment process was flawed enough to create an odd scenario: the hiring manager, desperate to fill an urgent role, offered Felix a different position on a whim. There was no thorough assessment, proof of relevant experience, or evaluation of his potential to grow beyond the immediate crisis.

Small parentheses: We could talk for a long time about the damage caused by the hiring manager -who has urgency and bias to cover position- having the authority to decide how candidates are evaluated, or being the only person evaluating them, or having an unbalanced weight in the decision. We could also talk forever about the compound effect of hiring managers or interviewers who rely on their intuition, don't follow a well-established calibration framework, or plainly are not good at interviews. It is like not knowing medicine or chemistry, not believing in the professionalization of medicine, and choosing our own medicines based on how the box looks. 

Moving on to Felix, he accepted the role, which involved leading a small team. As time passed, the team expanded. Felix built rapport with his boss not through calculated strategy but by being overly obsequious and compliant to the point of insincerity. He also turned a blind eye to several wrongdoings, further endangering him to his superior, who favored such behavior. This lack of authenticity and integrity damaged his credibility among colleagues but secured his favored status with his boss.

One more small parenthesis: We can refer to a couple of the leadership rules from Maxwell’s 21 Irrefutable Laws of Leadership, such as the law of magnetism (we are surrounded by people who are just like us; for example, if we like to lie about deadlines, we like people who will cover up our lies, not people who will choose honestly) and the law of respect: a leader will have followers that are a bit worst leaders than themselves, people follow leaders that are better than them at least in some dimensions. 

Years later, the flawed promotion system worked in his favor, and Felix was elevated to a higher position. He had not garnered respect from his peers, had no genuine followers, and lacked the trust of key stakeholders. To an observer, his rise seemed entirely accidental rather than a result of deliberate effort. Felix was not a cunning strategist -and one can respect those people- but he was merely someone who ended up in a prominent role by sheer chance, without any intentional planning or skillful maneuvering. Nothing he did to account for it.  

Does it matter?

It does. People will angrily respect intentional achievements -even when delusional or dishonest- but will be disgusted by people getting power over them not only without the right skills but also that connecting the dots, there is nothing to learn from it, nothing to admire from it, and nothing to respect from it. And you should judge yourself in the same way.

No matter where you go, make sure that you are the one doing something to take you there.

Confidence and entitlement

You are in a certain position. You may deserve it or not. Do you act confidently, as if you feel competent, or trying to project that image? Do you treat others like you deserve what you have and can act on it like you can get away with things because of your position or your self-assumed superiority?

Competence versus confidence is a long discussion topic itself. Confidence can be intentional or can be a function of a lack of self-awareness and other factors, probably just an expression of it. For the current discussion, I'm less interested in it. 

Entitlement, however, is a behavior that astonishes me, so I’m more interested in going deeper into it. Both in the cases of actual competency or not, entitlement is a behavior that I believe is probably founded as well on lack of awareness, but -this is my opinion, and I’m not a psychologist- probably with a more significant contribution of more fundamental personality traits:

• Low Agreeableness often leads to less empathy and more competitiveness. People with low agreeableness show less empathy and can be more antagonistic or skeptical of others' intentions.

• Low Conscientiousness. Conscientiousness is associated with discipline and diligence, and while high conscientiousness contributes to being focused and stubborn, entitlement may correlate more with low conscientiousness, especially if the individual lacks the discipline to adhere to social norms or the diligence to recognize others' contributions and rights.

• Maybe some aspects of high Neuroticism, people high in Neuroticism may sometimes feel that they are owed more by the world or may use entitlement as a defense mechanism to cope with their emotional instability.

• And probably low Openness. This trait involves imagination, creativity, and Openness to new experiences. It may have less correlation, though individuals who are low in Openness might be more rigid in their beliefs and expectations, which could manifest as a sense of entitlement.

Each case can vary significantly, however, and the interaction of these traits can influence how strongly the entitlement comes across. Oftentimes, a lack of social and professional experience or awareness also helps. We can undress all this a bit: this is what I call being a junior in life, together with being an idiot. 

It reminds me of this case: a few years ago, a person in my team was hired with a very high salary despite being quite junior. He spoke highly of himself and knew all the new technologies. A few days into the job, it was clear to everyone else that he was not as good as he believed. Not only that, but also he was looking at others over the shoulder, being lazy, and not pushing himself too much. One day, he mentioned in a relatively private forum, “If they pay us so much, it is because we are very good.” Or something like that. Pretending that getting a big salary validates that there is an actual reason for it, that your big salary justifies by itself that you deserve a big salary. Spoiler alert: It doesn’t.

The guy was fired shortly after -thus, his salary went to zero-. I kept that in my mind for years to come as a perfect expression of entitlement, lack of self-awareness, and ignorance of the rules of the world that include that others -and not only yourself- will evaluate you and will have a say about your worth. 

Be confident, but don’t act entitled. First, it makes you an idiot, and people don’t like idiots. Second, you will feel like a double idiot if one day you learn you were not as good as you thought. 

And that takes me -finally- to the main point of the discussion. 

Actual competence and awareness

You are in a certain position. Are you competent at it? Do you know if you are competent, and does your perception match reality? In other words, are you self-aware of your own competency?

You may ask, does that matter? Is it not enough to be in a high position? It matters, intentionality matters, and awareness is the foundation for success.

There is no shame in lacking competency, but there is shame in pretending. Furthermore, it is pathetic not even to be aware of what you are.

This is the story of Blunder Boswell. Blunder was placed in a position of power without the necessary skills, and his lack of self-awareness prevented him from recognizing his incompetence. 

Frequently, leaders possess authority over areas they are not experts in. As someone’s authority grows, his or her authority spans new areas, and the broader the breadth, the shallower the depth. The more responsibilities a leader has, the greater their authority but potentially the lesser their specific expertise. This is not bad, but it requires a very precise level of self-awareness to know when to talk and when to, well, shut up.

If this awareness does not exist, and a leader gets confused and thinks that because they have authority over something, they need to intervene, it results in them speaking into areas beyond their competency and trying to direct more knowledgeable employees, which undermines efficiency, undermines morale, and makes the leader look bad and lose trust from everyone around.

Blunder Boswell was leading his whole department, which was next to a hundred people. He was very less experienced but had accidentally landed in a position that gave him high responsibility. He was so out of his area of expertise -if he even had one- that he couldn't even see that his knowledge on the matter was not relevant enough to even get an entry-level or mid-level position. But he believed that being the boss somehow made him the smarter person in the room. 

This is something that always strikes me. How very little you have to know about something to think that the others cannot possibly know more. When you know so little, your world is so small that you cannot even imagine there is anything beyond what you can see. 

Like this, Blunder used to tell more competent people than him what to do. He used to exercise authority in areas where he was incompetent and made any form of work nearly impossible. His team members wished he had stopped talking about things and let them do their work, and his colleagues and stakeholders dreaded interacting with him. But he just kept acting entitled and reminding others explicitly of his authority in the matter as a function of his position. 

Blunder lost trust, created enemies, and was unsuccessful in everything he owned. This was not because he was not competent but because he was not aware. 

So, what is the conclusion?

It is okay to be incompetent, but you must intentionally withdraw from executing your authority and decision-making and trust your teams or other teams to handle matters within their specialized fields. This will enhance overall organizational effectiveness, trust, and employee happiness and will probably allow you to survive when things get serious. 

But for this, you need to be self-aware. If you are not competent, and you are not aware, and yet you are in a position of responsibility, then I’m afraid that faster than you know, you will destroy what you took so little effort to build.

You had an opportunity that, accidentally or not, took you to a position that you don’t deserve. And that’s okay. That alone can be a success story. But if you don't distinguish having the position from having the competencies required for the position, if you think that the position is what gives the competency, if you know so little that you cannot even understand the skills that you are missing and others under you or around you have, then you are delusional and ignorant. 

If you don't know there is a difference between having authority and having the core competencies, you are condemned to fall because people can follow unfair power, but they rarely follow pathetic, unconscious, and unsustainable opportunism.