The use of AI in the workplace – from permitted to prohibited practices

AI is inundating the market with new HR platforms, solutions and applications that are available to companies. While the AI Act does not prohibit generally the use of AI systems in the HR space, it does nonetheless prescribe certain limitations and prohibits some practices. Companies, and especially HR managers, must be attentive to how AI is used for recruitment and employee management purposes.

Join us on this webinar to understand the do’s and don’ts on how to deploy AI systems in the HR space. We will focus in particular on certain prohibited practices as well as high-risk AI systems and their impact on employees.

Key Speakers:
Olivier Proust - https://www.fieldfisher.com/en/people/olivier-proust
Katharina Weimer - https://www.fieldfisher.com/en/people/katharina-weimer

#AI #Data #Privacy #Employment #HR

Transcript

0:06 · good afternoon everyone or good morning depending on where you are located welcome to this uh Phil Fisher webinar on the use of AI in the workplace from permitted to prohibited practices my name is Olivier P I’m a partner in the teeken data Department in our Brussels office and I’m delighted to be joined today by Karina bimer who partner in the teeken data team in our Munich office hello

0:39 · everyone so this is part of our AI webinar series that we have been running uh since the beginning of of this year uh this is the third installment and as you can see we are running uh three other uh three new webinars including this one so there will also be another webinar on the 24th of October on the rights of the data subjects under the AI act and are they any different from the gdpr and we will have another webinar also on the 21st of November on AI

1:09 · regulatory enforcement is it the same as gdpr or worse and so hopefully you can join us for those webinars as well regarding the previous webinars if you have missed any of uh the ones we have run in in our part one uh you can find them all uh on our YouTube channel

1:30 · we have recorded all of our webinars there and also the webinars that were part of the second series so without further Ado what are we going to talk about today well um as

1:46 · uh mentioned at the beginning the focus of today’s webinar is the use of AI uh technology or AI practices in the workplace so we will start and have a look first at some of the prohibited practices and then are a few things in the HR context that cannot be done then we will focus on high-risk AI systems and look at the intricacies of deploying

2:11 · uh Ai and and the impact it can have on employees we’ll also talk about generative AI given that everyone now is super excited about gen Ai and seems to be using it more and more and of course we’ll have a look at the risks and enforcement in this context and what are some of the steps and actions that companies can take and with that I will pass it to

2:36 · Karina thank you Olivia um thank you for all being here today and listening in on our uh webinar first of all to bring this gentle reminder to you we wanted to give you the timeline of the AI act so that

2:54 · you can reflect on what needs to be done by what time and and that also gives you an opportunity to check back on your own progress and where you might fall within the different categories and by when you should be doing that um first of all as you know the AI act came into force in August of 2024 so very recently it entered into

3:25 · effect and um the first Milestone will already be reached in February of 2025 um that’s the provisions on prohibited practices as well as AI literacy obligations and the general provisions on scope and definitions that means that you should already be checking whether you’re um have already implemented any prohibited practices that you will have to refrain from using in the future so that you can indeed

3:57 · stop that in February 202 25 the next important Milestone is indeed already in one year’s time in August of 2025 that relates to the general purpose AI models the provisions here will come into play also the penalties will come

4:17 · into play and that of course goes hand inand with the national appointment of competent Regulators now if you’ve calculated correctly and you understand the intricacies that means you have about half year of the AI Act enforc without

4:37 · having any competent regulator so if you want to make any severe mistakes it probably pays out to be an early adopter and finish doing that by August 2025 when The Regulators are appointed and may actually take action then you will then have another half year uh another full whole year until August 2026 for the obligations on the highis

5:06 · AI systems listed in Annex 3 to come into force and another year for the next AI systems of the higher risk quality which are listed in Annex one so this is a quite a staggered approach um which should help everyone

5:24 · to comply and first of all to classify their own AI systems that they intend to use and then also to comply with the various requirements in the meantime as you may already know there is an AI packed which is already in force which enables companies to pledge that they are already in compliance with the AI act this was created so as to um help

5:51 · companies comply sooner if they want to which already May alleviate some of the burden can we go to the next slide please Olivier oh prohibit to dayi practices that’s what we’re going to talk about first because these are the ones that you’re going to have to stop using in February 2025 at the very latest first of all and now please remember we’re talking about AI in the workplace today so however you want to use it within the company first is the

6:24 · subliminal techniques these are set out in article five so generally the prohibited practices are set out in Article 5 of the AI Act and the subliminal techniques um are in um in number I think five and

6:43 · we’re looking at Subic subliminal techniques that um distort people’s behavior by impairing the ability for informed decision and free choice and of course it is also required to cause significant harm or po possible to cause significant harm here um the risk of the

7:05 · important adverse impacts on physical psychological health or financial interests also need to be present examples typical examples of this are um machine brain interfaces uh or virtual reality the subliminal components are often used in audio or image um files

7:30 · there video stimuli that persons may not be able to perceive straight away because they escape human perception um one example is also here in the image that we have on screen you may not see it directly but

7:46 · there is an A A subliminal image in this can and with ice on it and water on it uh and maybe you can read it or maybe it comes to mind when you just look at the scan many people just tell oh I desperately want a beer right now when they see this image um and that’s one of the typical subal techniques they’re beyond human perception um other

8:11 · categories here are manipulative or deceptive techniques that subvert or impair a person’s autonomy meaning their free decision making um their choices in a way that they are not consciously aware of this is where important to um notice

8:31 · that this is will be um prohibited in the future next uh slide please Olivier exploitation of vulnerabilities now leaving aside that employees are often as such a vulnerable group there are vulnerable groups spelled out by the

8:51 · AI act exploitation of the vulnerability of a person or group of persons due to age disability social economic situations um ethnic and religious minorities all these people are considered to be of a certain vulnerability and again the the AI is going to be

9:17 · prohibited where the objective or the effect is to once again distort or manipulate the behavior of these vulnerable groups where this causes or is likely to cause significant harm an intention to cause such harm is

9:37 · not needed as long as the harm results from the manipulation now we’ve got an example for you in the background and that is a a for instance a predatory Financial Service chatbot where you have an AI power chat board chatbot which is deployed by a financial services company to offer loans and credit PR products and this AI system uses Advanced algorithms to analyze financial data

10:06 · credit cards activities consumer habits and other data to really identify and Target individuals who might be financially vulnerable so here we are talking about already targeting individuals who are identified by the AI

10:24 · as someone who might be living in financial distress and now we’re exploiting said vulnerability because the chatbot is programmed to use manipulative language and pressure tactics to convince these vulnerable individuals to take out for instance a highin loan or credit product it might use language

10:47 · which suggests a particular urgency and might suggest that the financial situation could even deteriorate if the individual doesn’t take out the addtional credit and that is an example to show how an AI system might Leverage The individual’s economic situation and and exploit the financial distress of the individual next slide

11:16 · please emotion recognition systems in the workplace will also be prohibited an emotion recognition system is an AI system for the purpose of identifying or in emotions or intentions on the basis of their biometric data now to be more specific what article 5 prohibits is AI

11:41 · systems that are used to infer emotions of individuals in the workplace it doesn’t have to necessarily be if based on the biometric data but that adds to it if you use an emotion recognition system that adds to it but in generally um AI systems that infer emotions of individuals are already prohibited if it is not prohibited it might directly be

12:09 · considered highrisk AI that is for instance the case if it is uh used in marketing or targeted ads so not in an uh in an HR perspective as an example of how a company might be using such an emotion recognition system is an AI system that captures facial expressions voice tones and physiological signals if for example in

12:37 · a disciplinary meeting or in an annual review conversation in the HR place where you’re using cameras and microphones because often these now take place via um remote communication so we have possibly eyes tracking facial muscle movements voice stress analysis and similar factors and the AI system will then be able to infer the emotional state of the employee in

13:08 · this case probably nervousness possibly some anger or frustration confusion anything like that and then HR might be able to use this analysis to assess the employees credibility sincerity but also the way that person is comfortable

13:30 · and and might actually make a decision based on this and because the employee is again in this case very vulnerable and cannot object basically to to such an analysis such systems are prohibited straight away thank you next slide please Olivier social scoring is another um

13:57 · another aspect that is going to be prohibited social scoring is basically the evaluation or classification of individuals by public or private actors based on their social behavior with a social score and that can lead to some sort of detrimental or unfavorable treatment the reason this is prohibited is because such analysis and evaluation

14:24 · based on social interaction has an impact on the human dignity and values of equality and justice and of course it also is a great risk of discrimination dep if you make uh for instance promotions dependent on social interactions between people um there are for instance AI

14:44 · systems that could evaluate employees based on their interaction with colleagues as an example on on teams in the in the chat platform but also participation in teams activities social Gatherings if these were for whatever reason captured on on video their

15:04 · presence on LinkedIn and social interaction on LinkedIn could also be analyzed don’t want to give you any naughty ideas here because it’s going to be prohibited anyway but these are just the the possible scenarios that we were thinking of when we were looking at this the AI system can then assign a social score to the employees to evaluate the engagement in the company and TI that to

15:30 · promotion or non-promotion to other benefits and advancement in the company this kind of sums up the prohibited practices and with the next slide Olivier is going to indulge you in the highrisk AI systems yeah thank you very much uh Karina for this uh very good overview of of the prohibited uh AI practices now

15:58 · looking at the high risk this is the the other um big category under the the AI Act and where there’s going to be um you know the bulk of the work for a lot of providers and deployers of AI systems now let’s look at what the AI act say specifically about high-risk AI systems in an HR context and first of all it it

16:20 · does mention um Recruitment and I have to say that the the AI Act is quite both detail detailed and rather prescriptive detailed because it uses some um precise terms so it says

16:37 · that the AI if it is used for the recruitment or selection of natural persons will be considered as high risk also uh other actions are mentioned such as plac in targeted job advertisements or analyzing and filtering those job applications so in other words we can summarize this as AI being used as a screening process AI which is in

17:05 · obviously based on algorithms uh that can help you to scan resumés and cover letters identify for example some keywords and patterns that match a specific job description um for a position and open position within your organization and you can take it even a step further you can think think of AI that is going to be used to rank those candidates based on whether they match those criteria and

17:31 · those filters that you have set out whether they meet the required qualifications that you are looking uh for in in the recruitment process so um in other words AI that is used to evaluate candidates and this can also be used during an an interview uh assessment process this will automatically be considered um as high

17:57 · risk the other important section that is mentioned in the AI Act is AI that is used in the context of employee management and here again I have to say the AI Act is again both quite detailed and specific and prescriptive so if you are using AI to make decisions that are going to affect one way or another the terms or the conditions of uh the the work relationship for employees for example

18:28 · um if the de decision is to promote an employee or on the contrary to terminate an employees work contract or any other work rated uh situation or condition that that is going to affect the employee then such use of AI is going to be considered as a high risk but then

18:49 · it’s also interesting to look at some of the other specific examples that the AI gives for example allocation of tasks based on an indiv idual Behavior or personal traits or characteristics now why is this interesting because it is rather specific meaning that if you are relying on an AI tool to determine for example um who within uh your team is going to

19:15 · manage a particular project or who you are going to give a specific task or project to depending on uh individual behaviors or interactions or uh you know other factors uh that concern the individuals or the employees personality or or specific characteristics and and traits um that kind of use of AI will

19:39 · also be considered as high- risk and then last but not least and and I think this is one where we should expect The Regulators to be particularly attentive once they come into um once they are appointed and and start enforcing is the use of AI uh to Monitor and evaluate the performance and the behavior of uh employees in such relationships so here again if AI is used for employee

20:08 · performance reviews or appraisals um in a manner that is going to analyze for example work patterns their productivity metrics or provide feedback to uh the superior officers and possibly make recommendations on the basis of such analysis Rec recommendations uh to take certain decisions such as promoting an employee or making a salary adjustment or in the worst case scenario terminating an employees uh work contract all in all

20:41 · these situations this is likely to be considered as highrisk and so you are going to have to be particularly uh attentive and comply with the the provisions on high-risk AI uh systems in the AI act now I talked about what was quite

20:57 · obvious the there are also some areas that are a little bit less obvious this one is interesting education and vocational training now normally typically when you think about education and vocational training you think schools and universities were not

21:15 · necessarily thinking about employees and indeed the AI act says that if AI is used to determine access or admission or to assign natural persons to educational and vocational training institutions at all levels then it is high risk but and

21:32 · this is where I think it’s interesting the wording is a little bit broad because it also mentions AI used to evaluate learning outcomes and it’s not impossible that uh we we could imagine that well companies are indeed evaluating uh their employees on an ongoing basis including in the context of Professional Training continuous ongoing professional training throughout an employees career um and so there is

22:02 · an open question whether this provision uh under the AI Act was intended to apply exclusively to schools and and and uh universities education in the traditional sense or can we extend and

22:18 · and apply it a little bit more broadly also to the professional training so we don’t have an answer to this I think we’ll have to wait for The Regulators to explain and to give some guidance on this this but I think it’s worth mentioning and just keeping this in in mind because it’s not entirely clear um

22:35 · how this provision in what context specifically it will apply okay and then we come to another interesting one the remote biometric identification systems now this is a rather complicated area of the AI so I’ve tried here to break it down and to make it as simple as possible I think to

22:57 · start off with there are two distinctions that need to be made the first distinction is a distinction between two regimes so remote biometric identification systems can either fall into a prohibited category or into an authorized but high-risk category that’s

23:15 · the first distinction the second distinction is a terminology distinction between biometric identification and biometric verification which I will come back to in a minute but looking first at the two regimes so if you look first at what is prohibited the use of um remote

23:34 · biometric identification systems applied to individuals of course at a distance without any active involvement of those individuals by comparing their biometric data with the biometric data that is already contained in a reference database and if you are doing this in real time which means that there is an instant identification of the individual in a publicly accessible space and if it

24:01 · is used by for law enforcement purposes in if all of these conditions are met then that use of the remote biometric identification system will be prohibited but then if you look on the other side at the authorized but high risk you have the three same uh criteria the remote

24:20 · identification uh the the the lack of any active involvement of individuals and the comparison of their biometric data with biometric data in a in a database but if you are doing this in a purely private context so not in in a publicly accessible space and for the purpose of um verification then it is uh

24:41 · authorized but it will be high risk so what that means effectively is that uh it will really depend on a case-by casee basis we obviously we are thinking here about facial recognition technology because that’s basically what we are talking about but depending on on the context and how that facial recognition is going to be used it can either be prohibited or high risk now I mentioned

25:06 · that there is another distinction that needs to be made between identification and verification and here I have to say that the uh the distinction is quite subtle but when we talk about biometric verification we are not talking about what I just explained on the previous slide we are talking about Biometrics that is used for purely Authentication purposes to verify an individual’s identity for the sole purpose for example of giv access to a service or

25:36 · unlocking a device think about the the biometric system that unlocks uh your your mobile device or to give access to the individual to um restricted areas and premises at the workplace in that case that does not constitute a biometric identification it is considered as biometric verification and will not be considered as high risk and

26:00 · if you look into some of the recital of the AI act they give some specific examples where they these recital mention the company and Factory premises as well as office and workpl premises sorry workplaces that are intended to be

26:15 · accessed by the employees or service providers and are places that are generally not accessible to the public and are therefore excluded from the definition of real-time biometric identification in public spaces so we see here how the legislature um tried to at the same time circumvent and restrict the use of um biometric uh um uh

26:42 · technology and and facial recognition in particular in certain areas and and very specific contexts but at the same time um scoped out uh an area as an exception where it is used purely for verification purposes at the workplace and so this is interesting effectively what it means is that companies will uh be U allowed to continue to install um uh you know

27:10 · either facial recognition or other forms of biometric uh systems on their premises um and we can think of of for example nuclear plants or you know areas that are quite sensitive um and and where only um certain employees on the you know with with um authorized access will be able to enter right so having looked at the the high risk uh AI systems uh that are used in in the uh employment uh HR context

27:43 · it’s worth mentioning that um there are some exceptions that apply now these are the the general exemptions that apply to all types of high-risk AI systems that are mentioned in the ai ai act so not only those that apply in the HR context but it’s it’s uh worth mentioning them again um and essentially what the AI act says is that in for certain types of AI systems they will not be considered as high risk if for example the AI system is used uh strictly to perform a narrow

28:15 · procedural task U for example you know cross referencing some Financial transactions against counting records in order to verify the the accuracy or to improve uh the the result that of of um

28:31 · an action that has previously been taken by by an individual so through some actual human activity or it could be also to detect decision- making patterns or or deviations um so for example AI an AI system that analyzes a user’s transaction data to detect some patterns in spending uh Behavior now or or to

28:54 · detect some some frauds there are many other examples that we can provide uh here where where uh such AI may be used and and lastly AI um that is used uh purely to perform a preparatory task to an assessment um where um you know the that assessment is falls under one of the use cases that is listed in Annex three of the AI act so you have a few areas that are uh limited and and listed um uh in in under the the AI act um if

29:25 · you fall under one of these exceptions then it will not be considered as as high risk that being said as you can see they are quite restricted and quite limited and essentially it does come down to determining whether uh the AI

29:41 · technology is used without any human intervention or if at some point there is some form of human intervention and so uh to the extent that the AI is acting alone or that as an organization you’re relying exclusively on the AI te technology without any form of human intervention or review process then the

30:03 · the the chances are that you will not be able to rely on one of these Exceptions there is another uh exception which is in a way the exception to the exception if you understand what I mean which is that regardless of whether you fall under one of these four categories so a narrow procedural task improving the result detecting decision-making patterns or a preparatory task if if at

30:28 · any point you are profiling the individuals then automatically the high risk applies to your AI system again so what this means is that there is going to necessarily organizations are going to have to carry out quite a lot of uh risk assessment uh because the devil is

30:47 · really in the detail and what will matter is the specific context and the manner in which the AI technology is used and deployed and how it is used and to what extent uh there is a human intervention in the process all of this is going to have to be assessed um and and determined uh in in advance uh in order to to to know effectively if um such use of of the AI technology is high risk or

31:18 · not so that closes the this section on uh high risk and now I will pass it back over to Katarina who talked to you about the use of generative AI in the workplace thank you Olivier now generative AI in the workplace is is I don’t want to say common yet but more and more common almost every company us included is already using some form of AI and most of it is of course generative AI um and

31:49 · that is why we want to look at some of the details here um we’re looking at the next slide please where we have differenti at the general types of geni that companies are using and developing in in their in their field now you’re all aware that there’s lots of off the shelf offthe shelf software already that can be made available to the workforce one of the most most widely

32:19 · distributed is of course chat GPT or the competitor co-pilot they’re being used a lot um things like I companion but also um deepo is the translation service what we’re also seeing more and more being used um and and that’s not necessarily gen AI but it’s um it’s analytical AI is

32:43 · in conferencing tools where this is being used and the generative part is again that the conferencing tools automatically create protocols such as Zoom AI or also in teams um dysfunctionality is is being presented and it’s already widely being used what these systems gen AI systems and tools

33:06 · have in common is they’re off the shelf they’re not necessarily tailored to the company who’s using them um and they’re often also widely used in the workplace they can be used for lots of different functionalities and um they render lots

33:23 · of the tasks lots of the everyday tasks that we’re doing um a lot more more efficient and help the workforce in producing their their job results easier and more efficiently what’s often even better at this is the tools on the right hand side this the very specific gen tools um to specific tasks that we’re looking at some some of these are provided by

33:51 · external providers according to specifications of the company or sometimes if a company requires a very specific one and has a distinct need they will might even develop it themselves also depending on their internal capacities and competencies that they have ones that we do already see quite a bit of um as examples here are um customer relationship tool

34:20 · Integrations the there there’s lots of overlap and sales assistance we already talked about chatbots in the in the Finance industry before when we were talking about the prohibited practices of course this doesn’t always have to be a prohibited practice chatbots are

34:36 · widely used I’ve used them a lot um and I find um very often they’re more efficient than trying to get on the line with a telephonic support hotline so do feel free to make use of them when you encounter them it’s often a not so frustrating experience there’s also AI generated promotional content that um of course the companies try to or with which the

35:04 · companies try to Target Their audience better and then there’s customer and employee assistance so they can be used on on on both sides you try to uh assist your customers or you can even use kind of a chatbot or an assistant internally where you help your Workforce navigate for instance your it system or answer other questions that come up here in the in the workforce area these are the various geni tools

35:36 · that we can look at there are some requirements for certain AI systems which we’re presenting on the next slide in particular that and we want to highlight this for you because these are very important and and basically applicable to all the AI systems is transparency requirements this is one of the um one of the big lines that we

36:01 · always see with almost everything within the AI act transparency is one of the main motives of lots of the provisions in the AI act and we also see an interacting with the clients when they interact with their customers with their employees the transparency is valued by the recipient or on the other side these are we’ve given you here four examples for transparency requirements for specific AI systems um I’ve just mentioned the chat

36:34 · Bots as an example that is an AI system that interacts directly with people and if this is of course AI if I have an AI chatbot on the other side providers must inform the individuals that they’re interacting with a machine so you don’t have John do on the other side but you have the computer is talking to you um

36:59 · and you will probably eventually find out what you find preferable when you interact with these chatbots another transparency requirement is with generative AI providers must ensure that it’s it’s similar to to the chatbot providers must ensure that the output that you that you generate whether it’s audio image video text what have you that has to be marked

37:26 · as a a officially generated or manipulated I’m pretty sure that isn’t always the case looking at some of the LinkedIn contributions that one sees or other articles um where I’m not always

37:42 · sure whether the authors always indicate so correctly um but it has to be born in mind that you should be doing that another one that is again important in the in the workspace is the trans parency requirements in the emotion recognition systems the deployers hair

38:03 · must inform the exposed individuals of the operation of the system if you’re using emotion recognition systems you must inform the individuals and certainly in advance and don’t forget you also have

38:18 · to process personal data in compliance with gdpr now we’re always talking about the AI act but never forget that there is also lots of other regulations that you need to comply with anyway or in addition to the AI act you can’t just check off the box of the AI act for deep fakes deployers must

38:40 · disclose that content has been artificially generated or manipulated I understand that sometimes defeats the purpose of deep fakes but at least the law tries to rectify that next slide please now switching back br briefly to the high-risk AI systems um on which

39:01 · Olivier explained quite a bit there are obligations for providers and obligations for deployers let’s quickly sit with the obligations for the providers um those are detailed in article 8 and following the systems that they provide shall comply with the requirements in the AI act following article 8 now of course a risk management system that has to accompany the AI system um

39:28 · I would like to talk spend just a minute on the next bullet point data and data governance because this is so so so important also with regard to the quality of the AI system because in particular this this deals with data and data governance in article 10 and in particular if if those also used for training the AI this is a paragraph that requires that the data sets must meet certain quality criteria which are set out in article 10 um and the the

40:01 · training validation and testing data sets are must be subject to data governance and certain management practices design choices data collection processes regarding the origin of the data all of this serves to to basically

40:18 · ensure in article 10 number three that the data sets are relevant representative free of errors and complete this is is one of the most important paragraphs in my view when you’re looking at generally the quality of your AI system you also have to ensure that the um that the data sets

40:40 · the testing data sets are are um have the appropriate statistical properties regarding the persons or groups of person in relation to whom the higher risk AI systemm intended to be used and number five deals with biased ction and

40:57 · correction which is a major major issue in in the use of AI systems so please please please look at the quality of the data sets and how you can ensure that in addition technical documentation you have to have your Toms up to date you need to do recording um another one that is quite important is transparency and provision of information to deployers when I look at this in Article 13 of the AI Act it

41:27 · kind of sounds like you have to provide instructions for use that are similar to an Ikea manual um the information must be concise complete correct and clear now I know that the Ikea manuals aren’t always like that but we would at least like your AI instructions for use to be like that we also come to human oversight accuracy robustness and cyber security you must have a quality management system that of course then ties is back into the previous points

42:00 · and allows you to verify all these previous points and ensure that um documentation keeping and of course automatically generated logs and then the registration in the EU database that will come on top of it as the cherry on the next slide we also present you with some of the obligations for the deployers they are similar but not identical again you have to update your Tom you already have all your Ts for for

42:29 · gdpr purposes now you have to include the AI measures um in accordance with the instructions for use you remember your Ikea manual for the high-risk AI system um the human oversight obligation

42:46 · also applies to the deployers of course it’s even more important at the deployers that human oversight for the decision- making by the AI system is there the input data must be relevant and sufficiently representative here we’re trying again to ensure that the outcome

43:05 · is of good quality is without bias and can be used for the purpose the operation must be monitored you have to check on whether it continues to perform um in the way that you intended it to perform now also for the um for the

43:23 · provider to be able to do some quality management you have have to report your serious incidents back to the provider or the Importer or distributor and of course to your regulator once they are appointed logs have to be kept and here

43:39 · is now a very specific delectable especially for companies uh for countries such as Germany and France where you have a workers Council please please please involve your work as counselor or other representative body before you put into service a high-risk AI system you’re they they sometimes

43:58 · have codetermination rights they at least need to be informed if you don’t inform them and they find out afterwards that is not going to go down V individuals also need to be informed we had that regarding the transparency requirements two slides ago when if and when you subject them to highrisk AI systems you need to inform them of course a dpia may be required remember

44:25 · your gdpr obligation and of course we want you to cooperate with your authorities that always is a good thing next slide please and I think I’m handing back over to you Olivier thank you thank you kaarina and so indeed uh we are reaching the uh the last part of this uh webinar we’re going to focus a little bit on the risks and and enforcement um just a quick note to our audience to let you know that if you do want to ask a question uh you can do

44:56 · so you can type in your question in the questions box so please start thinking about uh any questions you may have um we will have a few minutes uh before the end of this session to to take them so if we sort of summarize um what we have seen until now and uh this is a very sort of simple way of presenting the the risk level of AI systems versus

45:21 · the probability okay so if you look at the prohibited AI practices the risk the level of risk is going to be very high because as you will see in in the next slide uh the highest fines under the AI act will apply to the prohibited AI practices that being said if you have done your job well and you have assessed and made sure that your organization is not doing anything that is prohibited then the probability of that risk materializing is going to be rather low

45:50 · for high-risk AI systems particularly when they are used in the HR space the risk level is also High because the fines and sanctions are quite high for high-risk AI systems particularly if you’re deploying them without complying with all of the obligations that kahina has just explained which apply to high-risk AI systems so there the probability could also be quite high and then lastly for generative AI such as

46:16 · the the chatbots or or tools that help you in your work uh the risk level is going to be relatively low um unless of course the the generative AI is used in a high-risk scenario so you always need to be careful whether uh the AI system falls under that highrisk category or not but assuming that’s not the case then the risk is going to be relatively low you really need just to make sure that you’re complying with that transparency obligation um but the

46:45 · probability um of of that risk materializing can actually be quite High why because uh the use of generative AI is much more widespread and and um and obviously you have your potentially your entire Workforce Now that is using or

47:02 · going to use generative AI tools so you want to make sure that um everyone within the organization is aware of the dos and don’ts whenever they are using gen um the fines well I think most of us are probably already familiar with them you have three categories of fines so as I mentioned if you place on the EU Market a prohibited AI system then you risk a fine 35 million euro or 7% of

47:30 · your company’s annual Global that is worldwide turnover um if uh you are in breach of your obligations for any high-risk AI systems and this also applies to the general purpose AI uh models that we we didn’t talk about in this session then uh you’re facing a 1550 million euro F or 3% of annual worldwide turnover uh these uh fines and sanctions can apply uh to all of the different actors in in the uh the AI ecosystem so

47:59 · the providers importers Distributors and deployers which means that the enforcement bodies can go after any of these uh organizations and they all have their respective obligations uh you know with regards to to high-risk AI systems so they all have a part to play in making sure that high-risk AI systems are used in accordance with the law and then finally 7.5 million or 1% of uh annual turn over

48:27 · if you are totally ignoring or or not complying with the requests um and inquiries coming from The Regulators now we did do a little bit of uh uh check-in with Karina on the position of the the The Regulators now we we focused on the data protection authorities obviously because the the AI Regulators are not yet in place um we only focused on three countries France Germany and the UK what we have found is that rough broadly speaking um most of

48:59 · the Regulators have at least these three have already issued uh some guidelines uh on AI but they tend to be at this point still General guidelines and focusing a lot on the interplay between the AI Act and the gdpr so for example uh the the Kil gives a few examples of

49:18 · AI that is used in the workplace but is not focus in specifically and and has not necessarily um looked at all of the the use cases where AI may be used in the workplace so a lot of the recommendations of these Regulators are going to be um fairly high level such as you know making sure that there is some human oversight um informing the employees going back to what Katarina was saying um and making sure that the AI is explained to employees and and avoiding uh bias um in the UK uh the IC

49:52 · also made some some rather practical recommendations making sure that uh companies are carrying out to dpia whenever the use of AI constitutes a risk for individuals once again informing them about AI particularly when the AI is used to make decisions that have an impact on the individuals and things such as U you know implementing the data minimization principle or techniques in order to to mitigate and reduce risk um Karina do

50:20 · you want to say a quick word on The Regulators in Germany if not you can see on the slide that uh in in Germany sorry I was sorry I was muted go ahead in in Germany um as you can see on the slide of course the the dear which is the d shots conference um and a kind of a a joint um discussion panel of the

50:49 · authorities they issued an orientation on AI and and data protection in general and also the the combination of these two and there are some points on AI in in the workplace and I’ve I’ve alluded to some of them before and so it they kind of stressed that you need to involve your DPO and of course the works Council um they’ve also recommend that

51:15 · if you want your employees to use AI you should be providing Company accounts and devices for that so that they don’t have to use the personal devices for instance a mobile phone for using the AI and then

51:30 · one of the important points that I that not only the dsk car stressed but also the Bavarian data protection authority is that you need to train employees on if when and how to use AI that is one of the key Focus points that all authorities like to see sorry Olivier thank you yes uh so

51:53 · just wrapping up um with a a few um tips and and recommendations on some steps and actions that you can take and and I think at least you know from what we are seeing at field Fisher uh many organizations are doing this already so I think you know a basic thing uh which is fairly straightforward uh is starting

52:13 · to map your AI systems um when I say straightforward I I mean it in in in terms of something that you need to do it’s maybe not as easy as as it may sound uh particularly in large organizations where there’s so much I that that is now being used but I think it is going to be uh necessary at least as a first step to to to understand uh what types of AI your organization is using or wants to acquire and start making those risk assessments the second thing is is what K highlighted the AI

52:44 · governance process now the term governance can have a different meaning in different organizations but I think um at a high level you should be thinking at least uh to to have a a sort of AI task force or or uh some key stakeholders who are um um properly trained on on AI and who are capable of making recommendations uh to to the management in order to to mitigate the risk and then start thinking about putting in place some policies um the AI

53:15 · literacy um Karina also mentioned this but upskill in your Workforce training um or or just raising awareness so through various campaigns this is also something that is very useful and then sort of link to that start thinking about uh Drafting and deploying guidelines or rules or policies for your Workforce especially with respect to the use of gen like always with new technology it’s very exciting at the beginning but especially if you’re relying on the free versions of um of

53:46 · the the Gen that is available on on the market that’s where there are risks and so at minimum you need to provide uh some guidance to your Workforce on how to use these tools in what context and the things they should avoid doing and then uh we don’t have time to develop this but this is an important area is start reviewing the terms of the contracts with your AI vendors whether it’s um the HR department or the procurement uh Department who is negotiating um the acquisition or or um

54:18 · licensing terms for various products and and AI tools that are going to be deployed or made available to your Workforce course you absolutely need to start looking at the the terms and and negotiating them and uh that’s where the determination of your role as either a provider or deployer of that AI is absolutely going to be key so with that

54:42 · um we still have a few minutes left to take some some questions and I don’t know Karina if you’ve had a chance to look in the question box um I I did have a look there are some questions let me pick out one um that came up asking um a dpia

55:03 · when required same as the gdpr requirements or enhanced the gdpr basically says that when you’re using new technology which is likely to result in a high risk to the rights and freedoms of natural persons so that implies that almost every high-risk AI system will require a dpia it will be difficult to argue for a highrisk GP uh AI system to get out of this for the other AI systems you will

55:37 · have to go back to gdpr and determine is it likely to result in a high risk even though it’s not a highrisk AI system and then to determine under gdpr requirements there is another question that maybe Olivier you would be happy to answer and that is do you have any other examples of of biometric identification

55:58 · versus verification yeah um that’s that’s a good question so I I did think about you know um what can be an example of biometric um identification in in the workplace um as a reminder um you know

56:16 · when you um um analyze this this legal Concept in opposition with verification we’re not talking about uh authen authenticating individuals verifying their identity but um comparing the biometric data of of uh employees compared to biometric data that is already stored in the database now I have to say that uh the I would expect

56:41 · that you know most of those use cases will will probably not apply so much in in the workplace but you can imagine for example that um some companies may may want to uh to maintain to have a a biometric uh system uh database and um if they have some some um facial recognition cameras that are active on on the premises uh they may want to then

57:08 · uh uh use that biometric system in order to identify um individuals who for example have committed a crime or felony on the work U premises and they are not doing this in real time because that would fall under the prohibited practice so if they are doing it post factum for example um as part of an investigation and they want to U find the identity of

57:33 · uh an individual who has committed a wrongdoing on the premises um and in that case the the company would uh uh would try and identify that individual through the biometric data by comparing uh for example um CCTV footage and images with the bi biometric data in a system maybe it sounds a little bit too far-fetched a bit too much big brother if you ask me U and hopefully companies are not really doing this but I don’t think we can exclude it alt together so

58:03 · that would be one example that I could give um to answer this question thank you Olivia there’s I think we have two more minutes for another question and this one is with regard to the training for employees is there any guidance on whether this should be all employees or just those using the AI in in my view it should be those using AI or those likely to use AI in the

58:32 · future oftentimes AI is not only rolled out to certain individuals especially when we’re talking about gen AI such as co-pilot or or chat GPT and then of course you want to train all employees it certainly doesn’t hurt that I I don’t have specific guidance on this from The Regulators but it certainly doesn’t hurt to train them because might be doing it on their own without you even knowing it that’s always a risk because employees do these things as you probably know um

59:05 · um Karina there’s maybe very briefly one last question that I did want to answer in a corporate s is a company if sorry if a company makes a third-party high-risk AI system available to its Affiliates does it become a provider of a highrisk system really really good question now with with some of my associates we are right in the middle of um assessing those kinds of of scenarios I what I can say is that it it’s not it’s not easy because it’s not always straightforward um the fact that you are making available a third-party high-risk

59:38 · AI system so I understand this as a highrisk system that has been developed by a third party um um you know does not necessarily um or automatically put you in in the category as a a deployer um it

59:53 · really depends on on whether you are instructing that third party to develop an AI tool or system on your behalf or if you are requiring something acquiring something for example off the shelf um the fact that you are then making it available to your Affiliates I I’m not

1:00:12 · 100% certain that this is really the determining Factor uh in order to to to to decide whether you’re acting as a provider or or deployer um it’s it’s rather um I mean I guess the the question comes rather from the the making available uh and and those are the terms under the AI act um but but I

1:00:35 · think you you you need to me what what is probably important here is is trying to assess at the start um you know who who is the developer of the AI because the provider is really um initially the one who has developed that AI but um not not an easy um question to answer I have to say because of the the the the difference scenarios that are possible and so I think it will really depend on on a case-by casee basis um so I think we’ve reached the hour so on this slide you you’ll have seen that um if you want to to access

1:01:09 · more information more detailed information you can find it on our website we have plenty of resource resources there that are accessible to to everyone and um yes this uh webinar has been recorded so it will be made available on our YouTube channel uh similar to all of the other recordings of um the entire uh AI webinar series

1:01:30 · that we have run throughout 2024 so with that we are one minute past the hour so I would like first of all to thank Karina for co-presenting this webinar with me thank you so much it was a pleasure and thank you Olivia and with that thank you very much to our audience and we look forward to having you again on our next webinar thank you very much bye bye bye bye