Skip to end of metadata
Go to start of metadata

Overview

One of the common methods for passing parameters to scripts is via name-value pair or dictionary, which can make code easier to read and understand. This differs from many other languages which use ordinal parameters. We recommend name-value pairs as a best practice rather than other FileMaker methods analogous to ordinal parameters, such as return-delimited lists, due to a number of advantages:

  • The name-value pairs are not sensitive to the order of the pairs. Ordinal parameters must exactly match what a script is expecting.
  • Name-value pairs are more self-documenting. The names give clues to what the values represent or might be used for without necessarily having to read the script that will use them.
  • With name-value pairs, optional parameters can be left unspecified, whereas care may be needed to leave gaps for unspecified optional parameters in delimited lists.

Script parameters are the most common application for these structures, but they are useful for other purposes as well.

This best practice suggests only the names of custom functions for working with name-value data structures and how the functions should behave when used together. The particulars of implementation are not specified. However, a reference implementation is available in the FileMakerStandards.org GitHub repository. A different implementation, even one using a different data serialization format, that still follows the behaviors described below would still match this best practice.

Core Functions

These functions implement the basics of adding data to and retrieving data from dictionary data structures.

# ( name ; value )

The # ( name ; value ) function creates a name-value pair. A dictionary data structure can be created by concatenating several calls to #() as if they were plain text. Name-value pairs can be nested.

Named values can be over-written or effectively erased by concatenating a call to #() to the end of a dictionary using the same name and a different value. This works because the #Get and #Assign functions will always respect the last instance of a named value in a dictionary.

This last-value-wins behavior can also be used to set default values for optional parameters.

By placing the defaults before the actual parameters, any values set by the actual script parameter will override the defaults.

#Assign ( parameters )

The #Assign function parses a dictionary into locally-scoped script variables. The name from each name-value pair is used as the variable name, and the value from each pair is used as that variable's value.

The #Assign function returns True (1) if there was no error detected while assigning the values to variables, and returns False (0) otherwise. If there was an error detected, FileMaker's error code is assigned to the $#Assign.error variable.

#Get ( parameters ; name )

The #Get function returns a named value from a dictionary.

Unlike the #Assign function, #Get will not modify any variables. This can be useful when a value only needs to be used in one calculation, or to assign a value to a variable with a different name.

Utility Functions

These functions are less fundamental to working with dictionary data structures than the core functions, but experience has demonstrated that this functionality can be indispensable in practical applications.

#AssignGlobal ( parameters )

The #AssignGlobal function parses a dictionary into global variables.

The #AssignGlobal function returns True (1) if there was no error detected while assigning the values to variables, and returns False (0) otherwise. If there was an error detected, FileMaker's error code is assigned to the $#AssignGlobal.error variable.

#Filter ( parameters ; filterParameters )

The #Filter function returns a dictionary containing only those name-value pairs where the name is included in the return-delimited list filterParameters.

This function can prevent an "injection" of unexpected variables that might cause problems.

#GetNameList ( parameters )

The #GetNameList function returns a list of names from all name/value pairs in parameters. This is useful when you don't know what names exist, and you want to iterate through all the name/value pairs that exist.

#Remove ( parameters ; removeParameters )

The #Remove function returns a dictionary containing only those name-value pairs where the name is not included in the return-delimited list removeParameters. This is complementary to the #Filter function.

ScriptOptionalParameterList ( scriptNameToParse )

The ScriptOptionalParameterList function parses a script name, returning a return-delimited list of optional parameters for that script, in the order they appear in the script name. This function assumes that the script name conforms to the FileMakerStandards.org naming convention for scripts.

This is useful to generate the argument used by the #Filter function to restrict variable assignment to parameters actually accepted by a script.

 When the scriptNameToParse parameter is empty, the function will use the current script name.

ScriptRequiredParameterList ( scriptNameToParse )

The ScriptRequiredParameterList function parses a script name, returning a return-delimited list of parameters required for that script, in the order they appear in the script name. This function assumes that the script name conforms to the FileMakerStandards.org naming convention for scripts. This is useful to generate the argument used by the VerifyVariablesNotEmpty function to validate that all required parameters have values. When the scriptNameToParse parameter is empty, the function will use the current script name.

 

VerifyVariablesNotEmpty ( nameList )

The VerifyVariablesNotEmpty function returns True (1) if each of the parameters in nameList has been assigned to a non-empty variable of the same name. The function returns False (0) if any variable defined by nameList is empty. This is useful for verifying that each parameter required by a script has been successfully assigned before proceeding.

Array functions

A standard format for array's has not yet been agreed upon, so a page has been created for it in the Proposals section: Custom Functions » Arrays

Legacy functions

Deprecated

Icon

The functions below this point have been deprecated in favor of the more efficient functions above. Use of these functions is optional, but not recommended.

These functions are documented for backwards compatibility. These functions are all equivalent to simple combinations of other functions, which can make the functionality more self-describing, especially for developers who may be unfamiliar with them.

#AssignScriptParameters

The #AssignScriptParameters function will assign all named values in the script parameter to local script variables of the same name. If any parameters indicated as required by the script name are empty, the function returns False (0); the function returns True (1) otherwise.

A combination of the #Assign, VariablesNotEmpty, and ScriptRequiredParameterList functions is the preferred way to replicate this behavior:

This approach enables greater flexibility in defining what variables are required and how those variables are assigned.

#AssignScriptResults

This function is exactly equivalent to this calculation:

#GetScriptParameter ( name )

This function is exactly equivalent to this calculation:

#GetScriptResult ( name )

This function is exactly equivalent to this calculation:

  • No labels

68 Comments

  1. I can post a demonstration file if that would help anyone's evaluation of the proposal.

    1. It would be cool to see a file of this. Can you email it to the list. I'm pretty sure I could adapt to this.

      If we're open to discussion on the semantics, I'm curious about your thoughts regarding Assign vs. Declare vs. Create vs. Generate, etc.

      I can see how Assign applies to whether the variable does or does not exist vs. the others. You can assign something to nothing and also assign something to something that exists.

      Regarding the #AssignScriptParameter function. I'm left wondering why it's not #AssignScriptParameters in the plural or something more applicable (see below).

      Myself I've used Ray's naming of DeclareVariables and my most recent choice of the more simpler CreateVariables. I would personally opt for naming which more tightly reflects what is happening as an end result.

      Where #AssignScriptParameter is not the result. The result, as indicated by your comment...

      ...to declare and populate a series of locally-scoped script variables...

      means that #SomethignVariables more clearly indicates what the result would be - where Something is Create, Declare, Generate what have you. You're not Assigning a Script Parameter, rather creating variables - unless I'm confused about the description of the function.

      I like your idea of defining the naming convention and not the method of implementation. Good idea!

      1. I'll put a demo file together tomorrow.

        Regarding the semantics, I started working with the # prefix after seeing Richard Dyce's comments under Naming Conventions. (I've used "DeclareVariables" in the past, too.) For the functions that act on a parameter string, naming based on the source of input (assuming the effect of calling the function would be understood by convention and frequent use) was my first stab at it, since it's the main thing differentiating between the functions. (This is also why #ScriptParameter and #ScriptResult are both singular, after FileMaker's Get ( ScriptParameter ) and Get ( ScriptResult ), respectively.) It was concise; I liked it. (But I decided that #Parameter and #Result were unhelpfully concise.)

        But Arnold asked for verbs, which is reasonable enough. "Assign" came up first, so I ran with that. Now that "Create", "Declare", and "Generate" are also on the table, I still like "Assign". "Assign" works both ways; the functions are assigning variables the specified values, and they're assigning values to variables. "Create", "Declare", and "Generate" all only apply to the variables side of the equation.

        1. Anonymous

          Did the sample file ever get posted, and if so where?  I would like to look at it.

          Thanks!!

          1. Alas, that's one demo file I never got around to building. However, these functions are all in the Standards.fp7 file.

  2. The comments on the Naming Conventions page about scripts include some discussion of different preferred formats for passing parameters. This proposal (in its current form) explicitly does not specify an implementation of the format of the parameter itself, only of functions that create them. However, the particular implementation that I personally use produces the exact same parameter as the style Matt Petrowsky was suggesting (almost). So...

    ...produces the same text string as this:

    1. One thing I see missing from this is a "getter" function for reading from an array of parameters.

      We've got the # for setting, but we need something for getting. Essentially what we have here is more about dealing with tuples than just explicitly working with parameters. Having a setter and getter for these - in addition to the instantiation scripts - is desirable.

      1. Forgive me if this comes off as tautological. If you passed a script parameter like this:

        ...and you have a step like this at the beginning of a script:

        ...then you would "get" the value from parameter1 with this:

        I should clarify this in the spec. Having a dedicated #GetValue ( parameters ; name ) function may be something a lot of people are used to, but it's really not necessary.

        The fmp7script URL protocol only gave me more reason to like this, since variables defined for the script via the URL and variables created via functions handling the script parameter can be treated exactly the same way — scripts can accept both parameters and direct variables without any need to "handle" the difference. I know the standards don't prioritize the considerations of working with FileMaker Go, but I'm not going to complain if I can eat my cake and have it too. The implementation of #AssignScriptParameter I used in the script parameters demo file I uploaded should correctly acknowledge parameters passed directly as variables in addition to the conventional method, and I'm thinking that it's probably worth it to write that behavior into the spec, too.

        1. At least some clarification should be added as to why it's preferred to run the declaration call instead of pulling from the array of name/value pairs using a getter.

          I can see the validity in changing my thinking process of not using getters, however, I'd like to point out that the current thinking around this (at least as it seems to me) is just within the realm of script parameters. This is again, a good reason for a less specific naming oriented more towards the end result of variables being created.

          #AssignScriptParameter (being singular) is VERY context specific. In one my uses of this type of functionality. I store embedded local variables within an array of global variables for internationalization. Therefore, the creation of variables is not ONLY used when name/value pairs are inbound to the script. This same functionality is used within the script and within other calcs too.

          I don't think we're talking about just Script Parameters here. If we followed the DRY principle, we wouldn't want to create the same functionality when #AssignScriptParameters does the same thing as #DeclareVariables and some developers following these best practices think it ONLY deals with script parameters.

          Or is it your intention that it only deals with script parameters and the reservation of # is only for script parameters and not general solution tuples.

          1. #AssignVariables instead of #AssignScriptParameter ?

            We don't have to follow FileMaker's use of singular because if I had to guess, FMI choose singular because they follow the "ease of use" model for their software.

            Get ( ScriptParameter ) is easy enough for a newbie developer who only "thinks" you can pass one parameter - which is typically what a newbie wants - until they find out they want to pass more. The very nature of passing more than one parameter already implies plural.

            1. WOW! I think I see where I was confused. I wish we had a few more eyes on this one. I think #AssignText was what I was talking about.

              I re-read the spec and I think this is what I'm thinking.

              #( name ; value )
              #AssignVariables
              #AssignParameters
              #AssignResult

              If I was confused, I wonder if others might be (not that I should expect myself to understand everything on first read) (smile)

              My comment above about singular vs. plural does have some merit. FMP inherently implies singular now. Anything beyond singular is plural. (wink)

          2. If I only wanted to deal with script parameters and results, I wouldn't have bothered to include #AssignText. The value of a "getter" function that we haven't brought up is that it's usually assumed to be non-destructive. Any of the #Assign functions could wipe out the contents of a $variable, whereas a well-behaved getter function would just return the asked-for value and leave $variables alone. Some implementation approaches make the coexistence of #Assign and #Get hairier than others.

            I think I've convinced myself to agree with the plurality of the names. Named-value script parameters started as one approach to passing explicitly multiple parameters despite FileMaker only giving us one. (If the fmp7script URL protocol is a sign of things to come, that may change.) That origin is worth respecting in the names. I want to keep "Script" in the name, though; I don't want any potential ambiguity about where the input is coming from. Since #AssignText is the only one (unless we decide to add a "getter") that takes a parameter, I suppose it's reasonably obvious what the input is, and there's no need to name the function based on it.

            1. Yeah, I can see the argument for #AssignScriptParameters. I'm cool with that.

              My only remaining gripe would be #AssignText. What's getting assigned? Is the variable 'getting' assigned, or is the text (which is not always the case - sometimes it's a number) 'being' assigned to the variable. (wink)

              My point being - Text is very ambiguous relative to what is being done here. I can definitely see that some implementations will preserve data type - therefore making text only one of the various types being assigned.

              [ ...comment updated moments after being posted ] Duh, I see you had made the change to #AssignVariables. Nice, I think this is more descriptive of what is going on. Awesome spec!

              1. The value of using the verb "Assign" is that the answer is both; that's the point. There certainly are variations on name-value pairs that recognize the type of the "value" portion of the pair, but I have a hard time imagining a format for the combined name-value tuples themselves that can be represented in FileMaker as something other than a text string. (Well, I can imagine that, but it's needlessly messy; I can't imagine _why _you would want to.)

                1. I'm totally cool with what we've got here. I like "Assign", I like "Variables" I like "Parameters" and I like "Result". They all speak to what they deal with.

                  I say ratified!

  3. Anonymous

    I also use CFs for - but not exclusively - script parameters. Named parameters are useful in many situations. e. g. for a stop watch function (http://www.kegebein.net/blog/2010/06/unlimited-timer/). My parameter functions act very much the same way: you have one function to define a parameter (name-value pair), a function to assign all parameters to local variables, and so on.

    First, I like your proposal for a common set of functions but not a specific implementation.

    Second, I like the use of # as name for the function that defines a parameter. I use a different name, but it would be easy to change.
    Also, I like using # as the first character for the related functions.

    Third, I suggest more descriptive names for the other related functions:

    • I suggest the use of param as data type instead of string. In many other program languages string is synonymic for text.
    • Instead of function name #String( param ) I suggest the name #Assign( param )
    • Instead #ScriptParameter( param ) I suggest #AssignScriptParameter( param )
    • Instead #ScriptResult( param ) I suggest #AssignScriptResult( param )

    Arnold Kegebein

    1. Anonymous

      Of course, it should be #AssignScriptParameter and #AssignScriptResult without any function parameters. My mistake,  too fast with the copy/paste.

      Arnold Kegebein

    2. Sounds reasonable to me.

  4. Matt, I tried to move this page from Proposals to Best Practices, and I found that I "cannot move this page to another space because you do not have permission to remove it from this space." Do you have better luck?

    1. Try moving them now. I created a new group permission for editors and put you into it.

      1. Thank you. That worked.

  5. After thinking a couple of days about the function names suggested here, I have some concern with one function: AssignVariables( variables )

    1. The placeholder should be parameters instead of variables. The function is working with parameters as the title of this page ("Script Parameter Custom Function") implies.
    2. In the cf #AssignScriptParameters and #AssignScriptResults, the part behind #Assign describes where the parameters are coming from when they are assigned to local variables. With cf #AssignVariables the part Variables refers to where it is going to. This would be misleading or, at least, not following the same rules.
      Btw, should it not be #AssignScriptParameter and #AssignScriptResult (both singular), the same as in Filemaker's function Get( ScriptParameter ) and Get( ScriptResult )? (Wow, I am nit-picking again (wink) )

    Conclusion, I would suggest to use #Assign( parameters ). Here, the function parameter itself is the part that is assigned.

    1. We've gone back and forth on the names of these a lot. I started with the singular forms to be consistent with the FileMaker Get() function flags, but I went with plural rather than singular since it reflects that these functions are historically inspired by the need to pass multiple parameters to a script.

      I was never quite happy with the name of #AssignVariables, and no one else was happy with the previous incarnations of it either. I hadn't thought of #Assign ( parameters ), omitting the object of the operation from the function name. I like it. Does anyone else agree?

      1. My only gripe with the more simple #Assign (although I LOVE simple - as long as it's understandable) would be the readability.

        Granted, anyone using the Coding Standards and Best Practices here will know what it does, but from the standpoint of knowing what #Assign does at first sight, it's a bit vague.

        I can see the utility of personally standardizing on using the name parameters when expanding them into variables such that #Assign ( $parameters ) is highly readable, whereas in the context of #Assign ( $someRandomSomething ) could head down that path of being vague.

        I think what we've got here is a cross of a best practice and good habits, such as using #Assign ( $errorVariables ) vs. #Assign ( $errors ), where the former seems to make more sense to me and the later reads as if you're setting some errors.

        The only argument I can think of to keep "Variables" is that it represents what we're dealing with. Can we think of times when we wouldn't deal with $variables and would be using #Assign generically?

        I'm cool with #Assign, I would just make a tip or note just underneath the function and promote the use of very descriptive argument names.

        Speaking of parameters vs. variables. Should we head down the path of the even more programatically generic? #Assign ( arguments )? This deviates from the specific domain of "variables" and "parameters" and introduces a new term. Just thinking out loud here...

        1. # ( "Blah" ; "Blah de blah blah blah" ) isn't the most intuitive function name either. The definition of the #Assign function includes that it acts on locally-scoped script variables, but not necessarily that it acts on anything we could call a parameter, so #AssignVariables at least made sense in that respect. I suppose it could function outside the context of a script, but the limited applications I can think of probably aren't good ideas. Within a script #AssignScriptParameters is probably going to get used a lot more often, leaving #Assign as kind of an esoteric insiders' function anyway; so I'm less concerned about it being as intuitive as the others. A note about using intuitive names for variables passed as parameters to the function is worthwhile, though.

          We've discussed the possibility of a #Get ( parameters ; name ) function before, which would return the named value without setting (or potentially wiping out) any variables, which would make it a little inconsistent to talk about variables with the other functions. I don't see how "arguments" is any more generic than "parameters"; it's just the more common word in non-FileMaker circles for the same thing.

  6. Matt suggested earlier that #Get functions were missing from this list. #Get functions would return one value at a time from the string of name-value pairs passed to them, and the functions would not set a script variable. This makes name-value pair strings more accessible outside of scripts, and it's an alternative that will avoid over-writing any existing variables in a script — in case a script result from a child script might include a parameter with the same name as a variable in the parent script, for example. I haven't personally worked on anything that had this problem since this page was created, but the possibility bugs me all the same. I wrote some example functions to play around with while I'm thinking about this:

    My first concern is that this would bring the count of parameter handling functions to 7, which feels like a lot after getting used to 4. I suppose it isn't so bad when I think of it as a matrix instead: 1 function for creating pairs, 1 function with 3 variations for setting variables, and 1 function with the same 3 variations for retrieving values without setting variables. I'm still getting used to that.

    We discussed before whether the script parameter and script result versions of the #Assign functions should have a singular or plural name. We went with plural for historical reasons. However, these functions only return a single value, so only singular makes sense. If we add these, is that enough for the consistency argument to have more weight, and to switch the #Assign functions to singular, too?

  7. I was thinking about the practice of avoiding literal references when it occurred to be that named-values are leaving literal references to script parameters all over the place. Who's OK with that? Who's not OK with that?

    1. I think some literal references are certainly unavoidable. The reason the word "refactor" exists is because of literal references (smile)

      The Developer Assistant product from Dracoventions makes it really easy to find things within various areas of FileMaker.

      1. Some literal references may be unavoidable in some places, but passing multiple script parameters is not one of them. ¶-delimited list parameters just have to stay in the same order, and the author of a script can rename the internal references to the parameters whatever she wants. (Changing the order or inserting new parameters in the middle can create the same kind of problems, though.) With Quote() and Evaluate(), there's no loss of generality in what you can pass. This comes at the expense of the self-documenting nature of the calculation creating the script parameter (or result). We've tended to prioritize approachability over functionality so far, and named parameters are consistent with that. I like to revisit old decisions periodically, though.

        1. If I understand your post correctly, you're referring to the fact that renaming $param to $parameter within a given script introduces points of failure?

          Whereas with fixed order parameters (as with most other languages) the functionality is consistent and leads to fewer points of code failure?

          One thing we could do is add a Best Practice page outlining the advantages/disadvantages of fixed vs. name/value parameter passing.

          Given tools like Developer Assistant, it seems more of a policy issue with teams of developers that you must have a sequence of "refactoring steps" to follow if you're going to use name/value pairs. The code is more approachable, true, but at the expense of a higher degree of ensured execution.

          1. I like the idea of having a page comparing the different approaches, but I think that sounds like an Accepted Techniques page. Then the Best Practices page can be, "we know there are multiple good ways to accomplish the same thing, but this one's the favorite." Didn't we briefly mention the possibility of doing something similar with Key values?

  8. I was explaining these functions and how to use them to someone this morning, and it occurred to be that the use of #AssignScriptParameters may not be as readable as it could be:

    A clever developer unfamiliar with the function might occasionally make a leap of intuition to guess correctly that #AssignScriptParameters is comparing the passed parameters to what's required by the script, but it's far from obvious. The source of those requirements (the script name) is even less obvious. Of course someone could just open up the function and read the documentation; but if there's a way to get by without that, I think it would be worth considering. Consider this alternative:

    It's less concise, and the developer still needs to have a clue what #Assign might do; but perhaps it's a step in the right direction. Thoughts? Function name suggestions? I didn't use the # prefix since the RequiredParametersAreAssigned function would not have to parse name-value pair structures like the functions already on this page do.

    1. My personal opinion is that we are playing a balancing act within the realm of "readability" and self-documenting code (the ideal) versus unfamiliarity.

      I place less emphasis on the worries about a developer having to go into a function to learn what it does or what it expects. The learning process is a matter of understanding syntax. When things are ambiguous, you have to seek documentation or read the function.

      For example, I would be willing to guess that not many FileMaker developers know (or care) about the difference between + and - when it comes to method declarations within Obj-C. That's because it has to be learned.

      I think the best aid we can provide is the documentation. If a developer is worth their salt, then what we currently have is both concise and readable. I would prefer to stipulate that a comment be used just prior to a script step to make note that anything in the script name within parens are require params.

      On a totally different note. I have modified the # function slightly in my working repository for the standards. In fact, if you want to see all my working code then use that branch.

      https://raw.github.com/petrowsky/fmpstandards/working/Functions/%23.fmfn

      I modified the # function to accept the explicit declaration of a global variable. This leaves everything else intact but prevents me from making yet another function for dynamic global variables.

      Would anyone on this list, who gets the emails, like to do a web meeting and walk through the functions we have to agree on some more standard custom functions? Reply through the Google Groups email list.

  9. Dan Smith started a conversation about these functions with me yesterday, and I was inspired to do a rehash that addresses the concerns I've had about the current set-up. I've put the documentation of what I plan to do on GitHub, and I'll be loading the implementation there as I actually implement it. What do you think?

    1. How about #AssignWhiteList instead of #AssignSecure? The word "secure" can have many interpretations, but "white list" makes it clear that you are only assigning a list of accepted parameters.

      Should some note be added to the #() function to let users know that the name parameter cannot contain certain characters, and must be shorter than a specified length? I assume that local $variables have a length limit - but I'm not sure what it is off-hand. Also, should this function return an error, or nothing, if the name parameter is invalid?

      1. I like #AssignWhiteList. Would it be enough to say that names must be valid as Let() variable names, or should I state (after discovering!) what's acceptable? I'm inclined to let the error occur when an #Assign or #Get function is called out of laziness in implementing the # ( name ; value ) function, but if there is an error response, perhaps it should take the form "// Error ## ..." so as not to break the retrieval of other parameters when Evaluate() is used to extract them.

        1. Would it be enough to say that names must be valid as Let() variable names, or should I state (after discovering!) what's acceptable?

          Yes, I think that's acceptable. But, to expand on that; I think that however it's stated in the documentation should be how it's validated by the custom functions. So, if you test for existence of invalid characters and length to validate, then that's how the documentation should state it. So, I think it should be validated by wrapping in a Let() function and using IsValidExpression(), to match the documentation that says "must be valid as a Let() variable name".

          I'm inclined to let the error occur when an #Assign or #Get function is called out of laziness in implementing the # ( name ; value ) function

          I understand it adds overhead to validate in the #() function, but I think it makes the most sense to create the error when it happens. Also, as currently defined, the #Get() function would not produce an error, since it does not retrieve the value by using the Let() function.

          but if there is an error response, perhaps it should take the form "// Error ## ..." so as not to break the retrieval of other parameters when Evaluate() is used to extract them.

          I like this!  If there is an error, you can easily see it by viewing the dictionary as text. I think that comment line should include the provided name and value parameters.

    2. First off, let me state what I do like about what you've done.

      1. #Assign ( Get ( ScriptParameter ) ) is utterly obvious. I like it.
      2. #AssignGlobal is a nice addition. As well, not changing #Assign to #AssignLocal seems perfectly logical to me because #Assign is so much more common.
      3. I think the objective of adding a layer of security to parameter passing is a good idea! While not many FileMaker devs know you can execute any script in a FileMaker solution using a script triggering plug-in, having a filter on the inbound parameters is a nice security addition.

      Here's what I don't like.

      I'm not fond of "FSON" because the structure of a Let function has nothing to do with Object Notation. I would almost much rather say the data is in a "let" format or "LETS" (meaning Let Standarized). In this case, it seems like you're trying to come up with an acronym just to do it because it's "tech cool" (not wanting to offend you on this - just calling it how I see it). Simply saying we've standardized on the "Let" format would work for me. See my ADDITONAL comments at bottom. I've always favored FileMaker's native escaping such as Quote() with the Let() because it avoids escaping nightmares when trying to come up with parameter passing schemes like the Dict, et al functions from others. Staying native is cool!

      Because of the above stated "obviousness", I'm really not fond of

      ScriptRequiredParameterList ( scriptNameToParse )
      and
      ScriptOptionalParameterList ( scriptNameToParse )

      I think these add a layer of complexity to the amount (not the clarity) of the code required. Having to specify that many additional Get ( ScriptName ) functions (despite auto typing software) does not seem to provide additional benefits. I'm mentally seeking a personal example where specifying an "outside" script name might ever be required. Help me see the light if there is a practical example.

      For me, it's rarely, if ever, necessary that more than a singular reference to Get ( ScriptName ) should be integrated into a sanity check within the process. In the past, you may remember I had some type of variable set as a switch to either enforce the checking (or not) of the name - and paramter existence - of the script (e.g. $checkScriptName).

      What is the reasoning for breaking out Required and Optional into yet more functions?

      What about something that may seem like it's so FileMaker-like that it's crazy easy.

      #Assign ( ScriptParameters )

      Note that we're using the plural because FileMaker uses the singular (which we've discussed somewhere else I think). And furthermore, using a custom function of "ScriptParameters" would provide us with something such as this.

      #Assign ( ScriptParameters ( validate ) )

      Where we can now pass in any type of validation routine we want on the inbound parameters. This can be in the simple form of a Boolean for check existing or not (e.g. VariablesNotEmpty), or yet another custom function which does more extensive qualifications, such as white list filtering.

      For example, rather than

      #AssignSecure (
      Get ( ScriptParameter );
      ScriptRequiredParameterList ( Get ( ScriptName ) )
      & ScriptOptionalParameterList ( Get ( ScriptName ) )
      )

      I would much rather see something like...

      If [ not #Assign ( ScriptParameters ( validate ) ) ]
      # throw error
      Else
      # main script
      End If

      or

      Set Variable [$validVars; #Assign ( ScriptParameters ( validate ) ) ]
      If [ not $validVars ]
      # throw error
      End If

      The validate parameter could be anything from True to RequiredVars() to ParametersValidate() to SomeUltraCustomValidator(). Coming up with an integrated default like VariablesNotEmpty() being based on Get ( ScriptName ) seems rational but I wouldn't want to force us into a non flexible structure.

      The VariablesNotEmpty() seems like it would work but I know there are many times where an "empty" inbound parameter should be valid. Yet, when we use the logic of "required" within FileMaker, the variable MUST exist - even though $variable = "" does not. :(

      The bigger problem is that FileMaker does not distinguish between empty and null on variables. Sending a parameter in with nothing will evaluate to a non-existing null as opposed to an existing empty. I can recall this very situation where #AssignScriptParameters got me because my inbound values were something like

      # ( "foo" ; "bar" ) & # ( "baz" ; "" )

      and I wanted this to pass as legit.

      One thing we could do is set "baz" to something like a non-breaking space for the purpose of passing validation of existence and then destroy it as part of the evaluation.

      ON A SOMEWHAT, BUT NOT DIRECTLY RELATED TOPIC...

      #1: One other thing we need to establish in the standards is something like a $void variable. When I read your following code...

      Set Variable [$ignoreMe; Value:#Assign ( Get ( ScriptParameter ) )]
      If [not VariablesNotEmpty ( ScriptRequiredParameterList ( Get ( ScriptName ) ) )]
      Exit Script [Result:# ( "error" ; 10 ) // Requested data is missing]
      End If

      I'm was assuming that $ignoreMe was something relevant to the script as I read it. We should establish a standard for a non-script-valid $localVariable. The traditional $void won't likely pass with many FileMaker devs. Maybe the following?

      $%

      Although, while it has the benefit of being short, it's heading towards the arcane shorthand and ambiguity of Perl. You still have to decipher what $% means. I personally use $none or $nothing a lot. Suggestions on this?

      #2: This has set off on something I've been meaning to put into another proposal regarding the standardized storage of structured data within FileMaker. Essentially data stored into fields in the Let() format is the way to go, unless data typing is required, in which case the alternative would be something (actually probably) XML with data type attributes. Which FileMaker itself has.

      Just other stuff I've been thinking about.

      1. Oh, and before anyone replies about the suggestion of storing data in fields in the "Let" format, let me clarify this is not about "data" fields and is about storing stuff like "environment" data. Stuff such as window positions, last known records and layouts, state, user prefs, etc.

        The ability to Evaluate () a known format is just too powerful.

        Also, as a reminder to myself - and any dev who knows the power of Evaluate(). Since you don't want to expose a ton of Evaluates() all over the place (read: potential security holes) you want to validate that the first char of each line is a $ to prevent the potential security holes. (wink)

        In this case, we would want to validate that each inbound $var is equal to those within the Get ( ScriptName  ) so that $var snooping isn't possible when calling a Custom Function.

        If a solution is known to use the standards documented here and the Evaluate is open, then using it to gain access to data which is simply obscured instead of protected via data level access is obviously not desired. (sad)

        Of course, anyone with access to a Data Viewer can do enough snooping as it is.

      2. I'm not fond of "FSON" because the structure of a Let function has nothing to do with Object Notation

        I agree with this. If the term "FSON" is used, I would expect these functions to include other features of JSON, like array.

        Simply saying we've standardized on the "Let" format would work for me.

        Met too!

        ScriptRequiredParameterList ( scriptNameToParse )
        I'm mentally seeking a personal example where specifying an "outside" script name might ever be required. Help me see the light if there is a practical example.

        I was also wondering when, if ever, I would pass anything other than Get ( ScriptName ) to the ScriptRequiredParameterList function.

        What is the reasoning for breaking out Required and Optional into yet more functions?

        I believe this is for when a script takes more parameters than can fit in the script name, requiring the user to be able to specify the Required and Optional parameters when validating.

        #Assign ( ScriptParameters ( validate ) )

        The problem with this is the validation occurs before parameters are assigned to variables. I like the process that Jeremy proposed, which involves:

        1. Assign parameters to variables
        2. Validate the variables

        The VariablesNotEmpty() seems like it would work but I know there are many times where an "empty" inbound parameter should be valid. Yet, when we use the logic of "required" within FileMaker, the variable MUST exist - even though $variable = "" does not. :(

        I believe I've had trouble with this in the past too, but now that I think about it - couldn't this be handled by making a parameter that can be empty an optional parameter?

        #1: One other thing we need to establish in the standards is something like a $void variable.

        I actually like $ignoreMe, it made perfect sense to me the first time I saw it - it tells you exactly what to do with it! In the past, I've used $trash. I don't like $% because it's not clear to someone who's not familiar with it, whereas $ignoreMe, $trash, or $void are relativity self-explanatory.

        #2: This has set off on something I've been meaning to put into another proposal regarding the standardized storage of structured data within FileMaker. Essentially data stored into fields in the Let() format is the way to go, unless data typing is required, in which case the alternative would be something (actually probably) XML with data type attributes. Which FileMaker itself has.

        I think it's worth looking into incorporating data typing into these name-value pair functions. Has anyone ever tried this before?

        1. #Assign ( ScriptParameters ( validate ) )

          The problem with this is the validation occurs before parameters are assigned to variables. I like the process that Jeremy proposed, which involves:

          1. Assign parameters to variables
          2. Validate the variables

          Validating input before assignment would be part of my objectives as any validation after Evaluate() is now bypassed because we've passed through anything that could have been an injection. Parsing text is pretty easy. Even FileMaker will validate data before commit.

          Since anything inbound by #() is coming in pre-evaluated we can do pretty much anything to it. Run a recursive PatternCount on a white list or any type of clean up.

          While it would serve no real purpose that I can think of, we could even do a pre-eval prior to the eval that #Assign does and kill off any (or all variables) for #Assign to be the final sanitized input.

          On the topic of $ignoreMe, (to me) it's almost as if I was in front of you waving my hand and said "ignore me, ignore me". I can't. I just read you. $ignore as a prefix is also pretty common.

          $nonCaptured?

          1. I encourage a distinction between security and validation of adequacy. It's the difference between "I wont #Assign any parameters I'm not expecting" and "I do have all the parameters I am expecting." The former is what #AssignWhiteList ( ... ) (or #Assign ( #WhiteList ( ... ) )) is for, and the latter is what ValidateNotEmpty() and it's kin is for. I agree that injection should be controlled before Evaluate()ing a Let dictionary, but that's not what I was thinking of as "validation" for this purpose.

            Regarding $ignoreMe vs. $void vs. $none vs. $%, why should any one name for a throw-away variable draw any less attention to itself than any other? "$none" might be less distracting for one developer, but more distracting for the next. In any other context, I might say this is "just semantics," but this is a context where semantics usually matter. I don't like $void or $none, because they suggest that the value should be empty, and that something is wrong if they are not empty, which is not what I mean in cases where I use $ignoreMe — what's the right non-distracting name to see in the data viewer to mean, "No matter what value is assigned to me, don't pay it any mind"? Though $% may draw less attention to itself for some, it doesn't explicitly communicate that message. Perhaps I could learn that $% should be ignored by convention, but if I can have explicit communication and convention, that's gravy.

            1. #Assign ( #WhiteList ( ... ) )) 

              I like this idea even better than using an #AssignWhiteList() function. I think the addition of a function: #Filter( parameters ; nameList ) would mean #AssignWhiteList is not needed. I like the name #Filter, because it works the same as built-in functions Filter() and FilterValues(), only it works on a dictionary. Just to clarify, I see this function returning all parameters that exist in the nameList, and by parameter, I mean the equivalent output of #(name;value) function, for each name in nameList.

              I agree that injection should be controlled before Evaluate()ing a Let dictionary, but that's not what I was thinking of as "validation" for this purpose.

              I agree with this statement. I think the dictionary functions should be the ones to prevent variable injection.

              what's the right non-distracting name to see in the data viewer to mean, "No matter what value is assigned to me, don't pay it any mind"?

              I think this is the right question to ask. The variable names I think of after reading this line are: $ignoreMe, $ignore, $trash, $garbage, and $junk.

              1. I like #Filter. I like it a lot.

      3. Regarding preserving the data type, I think this can still be done with Let() formatted serialized data. In the #() function, just adjust the value as necessary:

        If (
            value = GetAsNumber ( value ) ;
            value ; // don't quote the value, which will (hopefully!) preserve it's type
            Quote ( value )
        )

        If the above doesn't work, perhaps wrapping the value in GetAsDate, GetAsTime, GetAsTimestamp, or GetAsNumber function will work?

        1. I'm working on this idea here: https://github.com/dansmith65/FileMaker-Techniques/blob/master/CustomFunctions/%23Name-Value/%23.fmfn

          Currently, I am storing numbers, dates, times, and timestamps as numbers - which works, but doesn't look right when you view the value as text. The problem is, I don't know how to problematically differentiate numbers, dates, times, and timestamps. Can anyone else think of a way to do this?

          1. Perhaps try something like this:

            Case (
            GetAsTimestamp ( value ) = value ; /* format value as timestamp */
            GetAsDate ( value ) = value ; /* format value as date */
            /* etc. */
            /* Else */ Quote ( value )
            )
            1. The problem with this is the following is true, which means most numbers would be treated as a Timestamp.

              GetAsTimestamp ( 10 ) = 10
          2. https://github.com/dansmith65/FileMaker-Techniques/blob/master/CustomFunctions/%23Name-Value/%23.fmfn

            Here is another attempt at preserving the data type. This time, after determining that the value is a number, date, time, or timestamp I test for existence of separator characters like \ or - for date or : for time. This list of separator characters will likely need to be adjusted to work with different date/time formats from different countries. Can anyone think of other common separators that should be added?

      4. I was never fond of the "FSON" name. It just seemed like the natural conclusion to how I describe why I like the format for use within FileMaker better than other alternatives by [imperfect] analogy with JSON. "Let format" will do.

        I included the scriptNameToParse as a parameter to the script parameter list functions in the rough draft because it makes those functions much easier to subject to rigorous automated testing, which I'm increasingly adopting for functionality intended for such broad applicability. Perhaps it makes sense for the production version to have an additional opening line commented-out from the test version:

        Let ( [
        scriptNameToParse = Get ( ScriptName ); ...

        The idea behind breaking required and optional script parameter lists into separate functions was to provide the most common inputs for the #AssignWhiteList and VariablesNotEmpty functions. As Dan points out, there are times when the results of the Script*ParameterList functions are not the inputs we want, so they act as shortcuts for the most common answer. They also make it easier to implement backwards compatibility for what I hope will become legacy functions as combinations of newer functions.

        Setting a precedent for broader #Assign input filtering flexibility with a pattern like #Assign ( #WhiteList ( Get ( ScriptResult ) ; $whiteList ) ) appeals to me. Part of the motivation for my rehash was to provide clarity by making a hard distinction between validating that the assignment of parameters to variables was successful, and validating that the values that got assigned are acceptable according to what a script is expecting — The former handled by the #Assign functions returning EvaluationError() codes and the latter by separate validation functions appropriate for different situations that simply return True or False.

        Dan beat me to commenting on the "#Assign ( ScriptParameters ( validate ) )" construction and how to handle validation of acceptable empty values. But if you wanted a validation function to accept the script parameter

        # ( "foo" ; "bar" ) & # ( "baz" ; "" )

        while rejecting

        # ( "foo" ; "bar" )

        a different SomeUltraCustomValidator() could be written to fit the bill. (And different versions of the function could work to validate before or after #Assignment.)

        We seem to be settling on three major things that we might want to do to a Let-format dictionary: 1. filtering, 2. assignment/retrieval, 3. validation. I have presumed that that order of operations was the most intuitive, but perhaps I'm wrong. At the very least, I figured validation should occur after assignment since the fmp URI scheme allows variables to be set while bypassing the script parameter: if a parameter is specified by a variable in a URI rather than a parameter, the assign -> validate order of operations has the same result whether a value was passed by variable or by script parameter.

        I'm actually leaning towards validating that each Let-format dictionary line does not begin with "$" to prevent variable injection when we do Evaluate() a parameter. I realize this is a departure from how the functions work with the format right now, but that's why we define how the functions interact with each other's results rather than the format itself.

        If I recall correctly, the # ( name ; value ) function as originally written by Matt did format the results to preserve numeric vs text types. I changed it to Quote() everything; perhaps that was short-sighted. I agree with Dan that I'd rather keep typed data in Let notation, and even expanding it to accomodate dates, times, and timestamps. If we are going to talk about using a non-FileMaker format, I'm putting my vote in for YAML.

        1. Regarding the ScriptOptionalParameterList ( scriptNameToParse ) and ScriptRequiredParameterList ( scriptNameToParse ) functions...

          What if sending these a blank parameter used the current script name? In which case, these two invocations of the function would be equivalent:

          ScriptOptionalParameterList ( Get ( ScriptName ) )
          ScriptOptionalParameterList ( "" )
          1. At first, I was a little put-off by this pattern; but then it occurred to me that FileMaker's own object ID functions work this way. If it's good enough for FileMaker, it's probably good enough for me.

      5. I just had a need to distinguish an empty parameter from a missing parameter:

        # ( "foo" ; "bar" ) & # ( "baz" ; "" )    // baz is empty

        vs

        # ( "foo" ; "bar" )    // baz is missing

        And I realized that this is already possible with the #Filter function:

        IsEmpty ( #Filter (
            Get ( ScriptParameter ) ;
            "baz"
        ) )

        This does require "baz" to be defined as an optional parameter, though.

    3. I spent some time trying to devise an #Assign function for dictionaries using Let notation that would prevent variable injection by an adversarial agent, but my adversarial half is winning. Presuming a format where all leading $ are removed for legitimate name-value pairs, here are some injections I came up with:

      // 1
      legit = "parameter";
      $attack = "injected!";
      // 2
      legit = "parameter";
      $$$$$$$$$$$$$$$$$$attack = "injected!";
      // 3
      legit = "parameter";
      $attack = "injected!";
      // 4
      legit = "parameter"; $attack = "injected!";
      // 5
      legit = Let ( $attack = "injected!" ; "parameter" );

      That last one will even circumvent the #Get function. Even before that point, closing these loopholes calls for code that doesn't share the elegance that Evaluate() originally brought to the table. I'm not sure securing Let notation against adversaries is worth it anymore. Don't #Assign or #Get dictionaries you don't trust.

      That still leaves securing against programming errors and backwards compatibility, which are both definitely worth it when possible.

      1. I'm not sure securing Let notation against adversaries is worth it anymore. Don't #Assign or #Get dictionaries you don't trust.

        I agree. I also did a few tests and had the same outcome as you. With that in mind, I think it's better to include the leading $ in the output of the # ( name ; value ) function. My reasoning for this is that it creates a clear pattern to test for "¶$" which can be used to differentiate a name/value pair from a list, as generated by this new function I wrote: #List. I think this will be an important step in converting name/value pairs and lists to/from JSON, as discussed here: Re: About.

        1. One thing I've found is that some folks like to add name-value pairs to a Let string by using the List function rather than basic string concatenation, which creates an extra return character between pairs. For the name-value pair notation, this doesn't really make any difference — "# ( ... ) & # ( ... )" is semantically identical to "List ( # ( ... ) ; # ( ... ) )" — but "List ( #List ; #List )" would not make an equivalent data structure to "#List & #List" as currently implemented. Is that OK, or at least tolerable?

          1. I don't think it's a matter of being ok or tolerable, but a matter of producing unexpected results. The #List() functions goal is to produce an ordered collection of values, so by definition the order of the value in the list is important. Since Using List() function instead of concatenation would add an extra return between each value, this would not be an accurate method of using the #List() function.

            An alternative is to ONLY support using the List() function, but I personally don't prefer that method.

  10. Dan and I tested and implemented (in that order!) a set of functions demonstrating the proposed revisions we've been discussing. The "Run Custom Function Tests" script in the FileMaker file there will verify the expected behavior as rigorously as Dan and I know how. I'm using the new set in some current projects, and I really like it. If everyone else likes it, I propose we update the best practice to use the new set.

    1. So... Thoughts? Thumbs up or down? Criticisms and revisions?

      1. Sorry for my own lack of response. I've been in the mire of doing drupal/php development on the new magazine site. I've not been able to take a closer look at this thus far. I would like to implement myself and I'm sure you guys have come up with some great revisions!

        I'll go take a look at the file right now.

      2. Gosh dang it! I wish we had regex in FileMaker. I'm going to have to raise some heck. (smile) It would be really nice if #AssignGlobal would format to our standards. (e.g. #AssignGlobal ( #( "foo" ; "bar" ) ) - does not currently result in $$FOO = "bar" (sad)

        One thing we could do is use the " = " (space/equals/space) as assumed and deconstruct the parameters into an array where we UPPER() all the var names.

        Worth it? Of course if parameters are not passed in via #() then we can't promise good evaluation - but then who using these standards wouldn't be using #() (wink)

        1. I'm fond of the interpretation that the output of the #() function serves as the definition of a well-formed name-value pair as far as the other functions are concerned, so of course I have little problem with relying on its particulars. If not for some of the things we've seen you doing with the functions, I would never have thought to propose an #AssignGlobal function. If you think it's worth it, go for it! I'll even modify the automated testing for #AssignGlobal if you like. (In the course of working on this, I've grown fond of TDD.)

          1. I must confess the test driven development is very nice looking - if not pretty daunting for any newbie. (wink) It's very thorough what you've done. Drupal has a testing framework that uses phpunit I think and coming up with something in FileMaker is very admirable! If only it didn't take double the time - more devs might see the value in it.

            Re: #AssignGlobal. I doubt I've got the time to implement myself. It would be a no brainer with regex, but I'll probably just rely on my own "good coding habits" and use uppercase when the # ( "VAR" ; "should be all caps" ) (wink)

            The data typing is a really nice enhancement. I went through most of the functions and aside from some documentation, like specifying that #Filter's second param is case insensitive (of course all of FileMaker pretty much is), I'd say let's roll with it and push it up.

            You can obviously make the changes here on the page and I'll add it into the Standards.fmp12 file.

            Do you want me to change the functions in the official repo? It seems like we're getting a bit spread out with all of us keeping different repos. It would be nice if we all did collab on the same official standards repo or worked off forks and did merges. We CAN use branches you know. (wink)

            1. Sounds like a plan. It might be nice if the tests of the functions went with the functions, since they can act as a benchmark for any future revisions. I agree about forks and merges; please pardon my inexperience with git.

        2. Anonymous

          Because FM is case insensitive:

          If you #AssignGlobal (#("foo" ; "bar" ) and then #AssignGlobal (#("FOO" ; "bat" ) it will show as $$foo as it already exists, whereas if you #AssignGlobal (#("FOO" ; "bed" ) when none exists you will get $$FOO

          John R

  11. Anonymous

    Jeremy

    how about adding an additional error state to the # function to disallow key with spaces

    PatternCount ( ~name ; " " ) ;
    11 ; // Name has spaces

     

    John R

    1. I don't agree that spaces in dictionary keys is or should be an error state. The #Get and #Assign functions both work fine with spaces in the name, and a $variable with spaces in it is syntactically valid in a calculation. The convention at FileMakerStandards.org is to not use spaces in variable names, but setting variables is not the only thing these functions are useful for, and I don't think users adopting some of the conventions and resources on this website should be therefore forced to adopt others. That's no way to win friends.

  12. Hey Guys,

    I would like to see the # family of custom functions pushed forward more than they already are.  The work done by Dan and Jeremy et al, is outstanding. The data type preservation, and Dan's new JSON parser are just killer. I would love to see these get wrapped into a mFM module.  ( Perhaps even making up the foundation of a core mFM module. )  Dan has already started one for the JSON parser.  I would like to propose that his module include all of the # CFs.

    In addition I would like to see the three Error CFs that have been kicking around be brought into the # family. They are so closely related they are hard to unravel anyway.  Here is what I have been calling them:

    • #IsError
    • #GetLastError(note)
      • 'note' -> (optional) some text describing the particular instance of this error. Very useful when tracking it down.  Sometimes "301 - Record In use" , doesn't help much when you are looking at a log.  But "301 - Record In use, Invoice Duplication failed" can tell you exactly where the problem was.
    • #CreateError(errorNumber, errotType, errorText )

    I don't care much about the names so I am open to other ideas.  ( #IsError seems right tho).

    The last part would be to see if we can get the #Function idea fleshed out.  This is not required but if somebody wanted to take it on I would be thrilled.

    I will be speaking at DevCon about Modular FileMaker and I would love to call out this proposed module.  FileMaker is in desperate need of a rich data format and the # Family is by far the best.  I'd love to see it spread.

    Thoughts? 

    I have new Google Group for mFM if you'd rather move the conversation there

    Todd