Issues related to Box Upload

Certified Associate Developer

Hello All,

 

I have created an application to upload file to box and accordingly create a shared link for each of the files being uploaded, To achieve this functionality in a more efficient way I create 10 app accounts and I'm using multithreading logic to use all these apps with respect to their current usage. Only issue i'm getting here is related to multiple request at a same time. If the number of requests are very less than the application works perfectly fine but when we are doing a load test with 500 requests at a time I'm getting below error

 

Current date\/time MUST be before the expiration date\/time listed in the 'exp' claim

I had discussion with box team but according to them its happening due to time synchronization gap between appian server and box server, I know this is not true because if such is the case then it should be happening even when the number of requests are very less. If anyone have work on this feature and have any thoughts or suggestions related to this then kindly help me to resolve this error. Thanks in advance

  Discussion posts and replies are publicly visible

  • Can you confirm the server time is in sync with time.gov?

    How are you load testing 500 requests?
  • 0
    Certified Associate Developer
    in reply to Shyam Bommakanti
    I have created a web api which basically calls a process model for attachment. Hence by calling this web api in a loop we are making 500 requests. Also as I said it works fine when the number of requests are less then do you think it had something to deal with time? If yes then also how can we make sure that it is consistent all the time? Is it something appian needs to handle, Right?
  • Hi,

    Before we can answer, could you clarify which error exactly do you receive with this code 500? Could you provide the logs snippets where the stacktrace is shown for the error and some screenshots of this process model plus the code of WebAPI which will allow us to understand the solution better?
  • Please check the following link on Box Community. It is exactly the error you are getting. community.box.com/.../17929

  • 0
    Certified Associate Developer
    in reply to Dmitry S

    Hey Dmitry,

    Below is the error that I'm getting

    Result: [statusLine=HTTP/1.1 400 Bad Request, statusCode=400, headers=[Date:Wed, 18 Jul 2018 21:57:10 GMT,Content-Type:application/json,Connection:keep-alive,Cache-Control:no-store,Strict-Transport-Security:max-age=31536000,Vary:Accept-Encoding,Set-Cookie:site_preference=desktop; path=/; domain=.box.com; secure,Age:0], contentType=application/json, body={"error":"invalid_grant","error_description":"Current date\/time MUST be before the expiration date\/time listed in the 'exp' claim"}]

    Error: [title=The external system did not understand the request, message=The request was missing required details or was improperly formatted, detail=HTTP/1.1 400 Bad Request]

    -> The Web Api is calling a process model using a!processStart and passing the folder name as a parameter.  Attached is the screenshot of a process model that is being called by the web api

    ->One more thing I noticed related to process model execution. As you can see inside the process model I'm writing some values to mysql table. those values are nothing but the start and end time of document attached. Inside the "Process Attachment" subprocess the actual document upload is happening where I have a logic to end the process model incase there is any issue with the api call but the time difference which I'm getting inside the mysql table is around 15 mins which means the process model is waiting in a queue when the number of instances are high and that seems to be the reason behind token time out error.

    -> I also visited the link that you provided me but according to them we need to adjust time on server. If there is any mis-match happening between the appian server time and the box server time than why it is working fine when the number of requests are very less?

  • I guess the problem is here:
    > time difference which I'm getting inside the mysql table is around 15 mins which means the process model is waiting in a queue when the number of instances are high and that seems to be the reason behind token time out error.

    Can the call be redesigned to be more efficient, i.e.
    1- all the logic can be placed directly to Web API code to be synchronous like a!writeToDataStoreEntity() and onSuccess on this you call a!startProcess, etc.
    2- use batch mechanism for documents call and processing, i.e. while calling WebAPI you provide some parameters for number of documents or so and process them in a batch afterwards.

    Let me know if that helped. Could you also provide the screenshot of Process Attachment process model?
  • One more option is to increase 'exp' claim value while retrieving JWT token as described here: community.box.com/.../52360
    but it's not supported by createtoken(privateKey, publicKey, sub, customClaims, kid, iss, alg, typ, aud, jti) Appian function as I see.
  • 0
    Certified Associate Developer
    in reply to Dmitry S

    Yes, createtoken() function doesn't support "exp" parameter and also it can not be more than 60 secs, So I guess that won't make any change here.

    Attached is the screenshot of inside processing of "Process Attachment". Let me explain the entire working

     

    -> I created 9 apps under box service account to accomplish multi threading logic.

    -> Out of 9 apps one app is only used to create folder and accordingly it will pass the folder id to successor nodes.

    -> The successor node will pick 1 document at a time and finds the usage of other 8 app accounts

    -> The account which is being less used will be taken for the processing and it will accept all the necessary authorization details for that specific app including folder name that was created

    -> By using all this information it will upload the document in desired folder.

     

    * The reason behind we are using multithreading concept is due to the fact that with single app account it was taking long time to process documents when the number of documents are high, Say 5 documents. So we have decided to use multi-threading where every document is being uploaded by different apps and results into less execution time. 

     

    -> Also its not the matter of process model timing getting stored inside the mysql table. That was just to see how much time this process model is taking for document uploads. After all we are not making this a web api thing, This concept will going to be used inside the main process model which handles job execution and related actions.  

    **** With that note I have one question in my mind. Suppose we got 500 requests at a same time for box upload that means 500 instances of a same process model will be called. Then how exactly the appian is handling those processes? Is it executing in parallel or sequential? Because as far as I observer, It seems as if the appian is showing us that all the 500 instances are initiated but in the back end its executing certain number of models at a time. As a result of which this token error arises because when the process is coming out of waiting list after certain amount of time then definitely its time is out of sync with the box time. Isn't it?